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FOREWORD 


The  analysis  represented  by  this  report  was  conducted  in  support  of 
the  computer-based  education  (CBE)  system  being  developed  at  the  Marine 
Corps  Communication-Electronics  Schools  (MCCES) ,  Twentynine  Palms, 
California,  under  the  command  of  COL  J.  DePrima.  An  earlier  draft  of  this 
report  was  transmitted  to  the  Marine  Corps  in  June  1979.  With  the  continued 
support  of  the  MCCES  Command,  the  CBE  system  will  be  operationally  imple¬ 
mented  and  could  develop  into  a  system  that  would  serve  as  the  standard 
for  computer-based  instructional  systems  in  the  military  and  civilian 
worlds. 

The  analysis  could  not  have  been  conducted  without  the  support  provided 
by  the  MCCES  staff  in  the  CBE  office,  particularly  Mr.  Joe  Henderson  and 
Mr.  Ray  Sorenson.  Special  appreciation  is  expressed  to  Major  W.  Parker, 
who  has  been  the  principal  developer  of  the  CBE  system.  Appreciation  is 
also  expressed  to  Dr.  Dewey  Kribs,  who  assisted  with  this  analysis  by 
reviewing  the  implementation  plan  and  providing  summary  comments  that 
were  incorporated  into  the  final  report. 


DONALD  F.  PARKER 
Commanding  Officer 


SUMMARY 


Problem  and  Background 

"^The  Marine  Corps  Conmunicatlon-Electronics  Schools  (MCCES),  Twentynine 
Palms,  California,  Is  developing  a  computer-based  education  (CBE)  system 
to  support  Its  training  requirement.  This  CBE  system  is  projected  to 
provide  computer-managed  Individualized  instruction  for  courses  concern¬ 
ing  the  operation  and  maintenance  of  communication  and  electronics  equip¬ 
ment.  The  CBE  system  staff  at  MCCES  has  initiated  procurement  actions 
to  acquire  the  computer  system  during  FY  1980.  The  Navy  Personnel  Research 
and  Development  Center,  San  Diego,  was  tasked  by  Marine  Corps  Headquarters 
(Code  OTT)  to  assist  MCCES  with  the  CBE  system  development. 

Purpose 

The  purpose  of  this  analysis  is  to  develop  an  implementation  plan  that 
will  be  used  by  the  MCCES  staff  to  aid  them  in  the  development  of  the  planned 
CBE  system.  The  CBE  office  needed  assistance  in  developing  the  strategies 
that  are  necessary  for  moving  the  instructional  concept  closer  to  the 
reality  of  an  operating  CBE  system. 

The  Implementation  Plan , 

The  implementation  plan  is  made  up  of  a  list  of  enabling  tasks  that  must 
be  accomplished  to  meet  identified  major  milestones.  Each  enabling  task 
has  been  prioritized  as  to  degree  of  criticality  for  successful  system 
implementation.  Personnel  needed  to  accomplish  each  task  have  been  iden¬ 
tified  and  classified  as  to  their  required  level  of  expertise,  according 
to  classification  levels  suggested  by  Marine  Corps  personnel.  ^ 

Additional  information  was  included  in  the  report  to  aid  in  system 
development.  This  information  includes  an  economic  analysis  of  the  rec¬ 
ommended  CBE  system  as  compared  to  current  lecture  instruction.  Other 
costing  information  was  included  in  the  form  of  sensitivity  analyses  in 
the  following  areas:  (1)  percent  of  training  time  reduction  resulting 
from  individualization,  (2)  time  to  develop  instructional  materials,  and 
(3)  effects  of  variations  in  student  load. 

Major  Recommendations  from  the  CBE  System  Analysis 

Although  the  initial  implementation  plan  that  was  developed  by  the 
Marine  Corps  CBE  staff  is  generally  good,  the  present  analyses  did  result 
in  several  major  recomsendations  that  would  affect  that  plan.  These  recom¬ 
mendations  have  been  incorporated  into  the  implementation  plan  provided  by 
this  report,  and  are  highlighted  below. 

1.  MCCES  should  lnmediately  contract  a  fully  experienced  consulting 
systems  analyst  to  aid  the  CBE  staff  in  the  design  and  development  of  the 
total  CBE  computer  system.  This  analyst  should  be  highly  trained  in  com¬ 
puter  systems  analysis  and  experienced  with  developing  large-scale  computer- 
based  training  systems.  The  estimated  cost  for  this  analyst,  for  approxi¬ 
mately  1  year,  is  $50,000.  This  analyst  is  needed  in  addition  to  the 
complement  of  analysts  already  scheduled  for  the  CBE  office. 
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2.  All  computer  subsystems  Intended  for  the  system  should  be  designed 
at  the  first  stage  of  system  design,  even  If  they  will  not  be  Implemented 
lsmedlately.  Failure  to  Include  all  subsystems  In  the  initial  design  phase 
will  guarantee  that  either  the  system  will  not  be  fully  functional,  or  the 
associated  costs  will  exceed  those  necessary. 

3.  To  assure  quality  instruction,  instructional  materials  should  be 
constructed  by  civilian  professional  material  developers.  Although  in¬ 
structional  development  costs  are  high,  at  least  the  first  two  courses 
scheduled  for  Implementation-Communication  Centerman's  Course  and  Fun¬ 
damentals  of  Digital  Logic — should  be  developed  by  contract  services. 

These  contracts  should  be  let  on  a  competitive  basis  to  obtain  the  lowest 
possible  price.  The  two  courses  thus  developed  can  then  serve  as  models 
for  other  course  materials  to  be  developed  by  Marine  Corps  personnel. 

4.  Staffing  of  the  MCCES  billets  dedicated  for  instructional  devel¬ 
opers,  systems  programmers,  and  operators  should  be  accomplished  on  schedule 
or  the  entire  system  will  be  delayed.  Deficiencies  in  staffing  the  instruc¬ 
tional  developer  billets,  or  delays  in  obtaining  contractors  to  develop 
materials,  will  significantly  delay  system  operational  capability. 

5.  Final  test  and  acceptance  of  the  vendor's  computer  system  should 
be  contractually  based  on  the  demonstrated  adequacy  of  the  system's  oper¬ 
ation  with  one  course  fully  implemented  and  students  under  instruction. 

This  demonstration  of  operation  is  the  only  way  that  full  system  operation 
can  be  assured. 

6.  The  following  courses  are  recommended  for  initial  Implementation  into 
the  CBE  system: 

a.  Communication  Centerman's  Course  (CCMC). 

b.  Fundamentals  of  Digital  Logic  (FDLC) . 

c.  Basic  Electronics  Course  (BEC). 

d.  Field  Radio  Operators  Course  (FROC) . 

e.  Radio  Fundamentals  Course  (RFC) . 

f.  Ground  Radio  Repair  Course  (GRRC) . 

For  these  courses,  all  current  Program  of  Instruction  (POI)  hours  that 
are  of  an  applied  nature  (e.g.,  laboratory,  practical  exercise)  should 
be  Implemented  for  management  by  computer  managed  instruction  (CMI)  procedures. 
Fifty  percent  of  the  knowledge  POI  hours  should  be  implemented  via  CMI, 
and  Che  remaining  5U  percent,  using  CAI,  to  minimize  instructional  develop¬ 
ment  costs.  The  50  percent  estimate  is  a  generalization  applied  across 
the  variety  of  courses  and  should  be  refined  as  materials  for  specific 
courses  are  developed. 

7.  Research  and  development  efforts  that  should  be  accomplished  to 
make  the  recommended  CBE  system  functional  include  the  following: 

a.  Development  of  a  CMI  Instructor  Training  Package,  estimated 
to  cost  approximately  $35,000. 

b.  Development  of  tne  evaluation  system  to  determine  CBE  cost- 
effectiveness.  Estimated  cost  is  approximately  $55,000. 


8.  Manpower  requirements  should  be  analyzed  to  determine  those 
needed  to  develop  and  operate  the  CBE  system.  This  analysis  should  be 
performed  after  the  specific  vendor  has  been  selected  and  the  specific 
system  acquired  through  the  procurement  procedures. 

Conclusions 

The  MCCES  CBE  staff  is  progressing  toward  development  of  a  computer- 
based  training  system  that  will  satisfy  normal  and  full  mobilization  MCCES 
training  requirements  in  a  cost-effective  manner.  Continuing  support  from 
MCCES  and  Marine  Corps  Headquarters,  and  the  willingness  to  commit  the 
necessary  resources,  is  needed  to  enable  this  major  instructional  systems 
development  task  to  be  completed  successfully.  No  matter  how  dedicated 
the  CBE  staff  is  to  this  effort,  substantial  personnel  and  dollar  resources 
must  be  committed  to  complete  the  necessary  tasks  of  instructional  system 
design,  instructional  material  development,  and  computer  programming. 

If  these  resources  are  made  available,  the  MCCES  CBE  system  could  become 
the  standard  for  large-scale,  computer-based  training  for  years  to  come. 

If  successfully  developed,  the  CBE  system  may  be  suitable  for  implementation 
in  other  Marine  Corps  training  locations.  This  CBE  system  also  has  the 
potential  for  providing  training  at  the  various  Reserve  Training  Centers 
within  the  Marine  Reserve  establishments.  The  decision  to  place  similar 
systems  at  other  sites  must  wait,  of  course,  until  the  MCCES  system  has 
been  successfully  completed. 
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INTRODUCTION 


Problem  and  Background 

This  report  represents  the  result  of  an  analysis  of  a  computer-based 
education  (CBE)  system  now  under  development  at  the  Marine  Corps  Communi¬ 
cation-Electronics  Schools  (MCCES),  Twentynine  Palms,  California.  It  is 
intended,  with  its  implementation  plan  and  economic  analysis,  to  serve 
as  a  working  document  to  assist  those  charged  with  the  task  of  turning 
a  concept  into  an  operational  computer-based  instructional  (CBI)  system 
that  will  manage  the  progress  of  almost  2000  students  daily.  These  students 
will  be  using  computer-managed  and  assisted  instruction  to  acquire  the 
skills  and  knowledges  associated  with  a  variety  of  communication-electron¬ 
ics  skills  used  by  the  Fleet  Marine  Force  (FMF). 

MCCES  is  the  largest  formal  school  complex  in  the  Marine  Corps.  The 
schools  conduct  approximately  45  technical  training  courses  largely  in 
the  areas  of  operation  and  maintenance  of  communications  and  electronics 
equipment.  MCCES  has  an  average  daily  load  of  1700  students  in  lecture-type 
courses  that  vary  in  length  from  2  to  35  weeks. 

As  electronics  technology  has  become  more  sophisticated,  the  equip¬ 
ment  used  by  FMF  personnel  has  become  increasingly  complex  in  its  operation 
and  maintenance.  As  a  result,  the  training  requirement  for  use  and  main¬ 
tenance  of  the  equipment  has  also  become  more  complex.  During  this  same 
period  of  increasing  technological  complexity,  training  support  has  de¬ 
clined.  Inflation  has  had  its  negative  effect  by  increasing  training 
operation  and  support  costs.  All  of  these  factors  have  placed  a  squeeze 
on  the  military  training  community.  It  is  not  surprising  that  the  training 
establishment  has  sought  remedies  to  the  dilemma  of  increasing  training 
requirements  and  decreasing  training  resources. 

MCCES  has  attempted  to  resolve  these  problems  by  turning  to  advanced 
instructional  technology  involving  computer-based  instruction.  The  initial 
attempt  at  incorporating  computer-assisted  instruction  (CAI)  into  the 
communication-electronics  schools  occurred  in  1973,  with  UMVAC  providing 
6'J  CAI  terminals  and  supporting  software  to  MCCES.  Despite  major  data 
transmission  problems,  due  to  the  off-site  central  processor,  sufficient 
data  were  obtained  to  justify  a  major  alteration  of  the  training  system. 

Since  the  off-site  processor  for  that  system  is  no  longer  available  to 
MCCES,  its  continued  use  is  not  feasible. 

Accordingly,  MCCES  is  proceeding  to  develop  an  expanded  computer-based 
education  (CBE)  system  through  a  major  procurement  action.  MCCES  estab¬ 
lished  a  CBE  office  that  has  guided  the  procurement  process  to  the  point 
that  a  "Request  For  Proposals"  has  been  released  and  vendors  are  working 
to  develop  systems  responsive  to  the  system  specifications  and  submit 
proposals.  At  present,  it  is  expected  that  the  contract  will  be  awarded 
early  in  FY  1980.  MCCES  is  working  to  acquire,  develop,  and  implement  the 
CBE  system  by  August  1980. 

The  CBE  system  envisioned  by  MCCES  will  provide  computer -managed  instruc¬ 
tion  (CMI)  features,  ultimately  for  all  courses.  More  importantly,  it 
will  provide  combined  CAI/CMI  instruction  for  selected  high  student-through- 
put  courses  in  order  to  capture  the  sizable  potential  savings  made  possible 
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by  individualized,  self-paced  instruction.  Students  in  courses  on  the 
CA1/CMI  system  will  receive  much  of  their  training  on  an  individualized 
basis  directly  from  the  computer,  including  receiving  instructions,  taking 
tests,  and  receiving  remedial  study  assignments.  Students  in  courses 
not  yet  individualized  will  take  tests  that  are  automatically  scored  and 
will  have  their  progress  monitored  by  the  computer.  The  CBE  system  will 
significantly  enhance  the  management  capability  of  MCCES.  This  should 
enable  the  school  complex  to  provide  better  training  management  with  fewer 
training  support  personnel. 

Purpose 

The  purpose  of  this  analysis  is  to  develop  an  implementation  plan  that 
will  be  used  by  the  MCCES  staff  to  aid  them  in  the  development  of  the 
planned  CBE  system.  The  CBE  office  needed  assistance  in  developing  the 
strategies  that  are  necessary  for  moving  the  instructional  concept  closer 
to  the  reality  of  an  operating  CBE  system.  This  need  for  assistance  was 
supported  by  Headquarters  Marine  Corps  (Code  OTT)  and  resulted  in  the 
analysis  contained  in  this  report. 


RECOMMENDED  CBE  SYSTEM  IMPLEMENTATION  PLAN 


Approach 

The  approach  taken  in  developing  this  recommended  implementation  plan 
involved:  determination  of  the  extent  of  CBE  system  development,  famili¬ 
arization  with  MCCES  training  requirements,  development  of  a  tentative 
CbE  system  implementation  plan,  and  then  completion  of  preliminary  cost 
analyses  to  determine  where  the  plan  needed  modification  to  result  in  a 
cost-effective  CBE  system  when  compared  to  current  lecture  instruction. 

It  was  our  intent  to  develop  a  system  plan  that,  if  followed,  would  result 
in  an  instructionally  successful  CBE  system  that  would  also  be  cost  effec¬ 
tive. 

The  implementation  plan  is  presented  in  detail  in  Appendix  A  as  Enabling 
Task  Statement  sheets.  Each  sheet  presents  a  separate  enabling  task,  and 
provides,  to  the  extent  possible:  beginning  and  completion  dates,  the 
performing  organization  element,  the  required  level  of  expertise,  and 
estimated  costs,  in  addition  to  the  task  statement.  Generating  estimated 
man-years  and  costs  for  many  of  the  enabling  tasks  was  beyond  the  scope 
of  this  project;  in  fact,  such  data  cannot  be  derived  until  after  the  com¬ 
puter  system  is  selected.  The  rating  scales  used  to  estimate  the  level 
of  expertise  required  for  personnel  to  perform  tasks  and  the  priority  level 
of  each  task  are  described  in  the  following  section. 

Prioritization  and  Required  Expertise  Rating  Scales 

The  prioritization  scale  was  developed  to  provide  an  indication  of 
the  relative  importance  of  each  enabling  task  in  assuring  the  initial  and 
the  full  implementation  of  the  CBE  system.  Meeting  the  objective  of  suc¬ 
cessful  initial  implementation  requires  having  CBE  systems  fully  designed 
and  substantially  developed,  but  only  partially  implemented  and  operational. 
This  degree  of  implementation  and  operation  requires  having  two  courses 
with  full  instructional  and  management  capabilities  on  the  system,  with 
a  third  course  having  student  registration,  automated  test  scoring,  and 
management  record-keeping  capabilities.  By  meeting  this  standard  of  initial 
implementation,  the  MCCES  CBE  system  will  have  demonstrated  its  full  CAI 
capabilities  in  two  courses  involving  maintenance  and  operation  training 
requirements.  Its  CMI  capabilities  will  be  demonstrated  in  the  third  course 
to  the  extent  of  being  able  to  track  the  instructional  progress  of  students 
in  other  courses  that  have  not  yet  had  the  full  benefit  of  individualized 
instructional  development.  Full  system  implementation  calls  for  all  six 
designated  courses  to  be  implemented  and  operated  in  a  CAI/CMI  mode,  with 
the  remaining  courses  being  operated  in  a  CMI  mode  supporting  student 
registration,  test  scoring,  record  keeping,  and  management  tracking. 

Each  enabling  task  listed  in  the  implementation  plan  was  rated  according 
to  its  relative  priority  in  terms  of  whether  it  is  required  for  (1)  initial 
system  implementation,  (2)  full  system  capabilities  but  not  full  implementa¬ 
tion,  or  (3)  full  system  implementation. 


To  provide  MCCES  with  information  about  the  level  of  expertise  required 
by  personnel  performing  each  enabling  task,  an  expertise  rating  scale, 
supplied  by  Marine  Corps  personnel,  was  applied  to  each  task.  The  com¬ 
plexity  of  each  task  was  evaluated  and  is  indicated  in  terms  of  the  extent 
to  which  MCCES  resources  can  accomplish  the  task.  These  levels  of  expertise 
and  their  corresponding  alphabetic  code  used  on  the  task  sheets  are  as 
follows : 

A — Can  be  accomplished  by  MCCES  personnel  presently  available  within 
currently  authorized  assets. 

B — Can  be  accomplished  by  MCCES  personnel  but  would  require  augmenta¬ 
tion  of  the  MCCES  T/0. 

C — Can  be  accomplished  by  MCCES  personnel  with  the  assistance  of  out¬ 
side  professional  consultants. 

D — Can  best  be  accomplished  by  a  professional  contractor. 

Enabling  Task  Statements 

Figure  1  illustrates  the  interrelationships  among  the  many  developmental 
tasks.  The  detailed  information  contained  on  the  enabling  task  statements 
in  Appendix  A  will  aid  the  CBE  office  in  developing  a  cost-effective  com¬ 
puter-based  training  system  that  will  satisfy  MCCES  training  requirements 
and  be  fully  accepted  by  the  Marine  Corps  training  community. 

The  enabling  task  statements  are  divided  into  categories  of  activities 
to  meet  the  following  seven  functional  objectives  that  are  necessary  for 
successful  system  development: 

1.  Enlist  the  participation  of  training  personnel  to  obtain  system 
acceptance . 

2.  Design  the  CBE  system. 

3.  Develop  the  computer  software  system. 

4.  Operate  the  CBE  system. 

5.  Acquire  the  CBE  system. 

6.  Develop  instructional  materials. 

7.  Train  CBE  instructors. 

Enabling  tasks  clustered  under  one  functional  objective  may  be  completed 
by  different  individuals  or  organizations,  but  the  activities  are  all 
directed  at  meeting  that  objective.  The  functional  objectives  are  listed 
in  the  chronological  order  in  which  they  would  occur,  except  for  objec¬ 
tives  (4)  and  (5),  which  were  reversed  to  keep  the  task  listings  consistent 
with  other  Marine  Corps  planning  documents. 


4 


Major  CBE  System  Development  Tasks  and  Dependency  Relationships 


Each  task  is  described  with  a  task  statement,  and  numbered  to  indicate 
its  sequential  relationship  to  its  complementing  task  for  that  objective. 

As  indicated  earlier,  each  task  is  prioritized  in  terms  of  its  importance 
to  achieving:  initial  system  implementation  by  August  1980  (Priority  1); 
full  system  capability  (Priority  2);  and  full  system  implementation 
(Priority  3).  The  level  of  expertise  required  by  performing  individuals 
is  indicated  in  accordance  with  the  classifications  suggested  by  Marine 
Corps  personnel.  Additional  relevant  information  about  required  expertise 
is  also  included.  Where  possible,  beginning  and  completion  dates  are 
provided,  along  with  estimated  man-years  for  the  effort  and/or  estimated 
costs.  Many  of  these  estimates  cannot  be  made  until  the  actual  computer 
system  is  selected.  Following  those  estimates,  comments  are  provided  on 
the  sheet  relevant  to  that  particular  enabling  task,  if  they  are  appro¬ 
priate. 

To  arrive  at  a  CBE  system  plan  that  would  be  as  cost  effective  as  pos¬ 
sible  when  compared  to  current  instruction,  it  was  necessary  to  alter  various 
phases  of  the  plan  and  to  perform  additional  cost  analyses.  As  with  most 
computer-based  instructional  systems,  there  are  major  areas  where  sizable 
savings  are  possible;  for  example,  reductions  in  training  time.  These 
savings  are  partially  offset  by  increased  instructional  materials  devel¬ 
opment  and  computer  costs.  Even  now  it  is  possible,  with  judicious  care, 
to  further  reduce  the  actual  cost  of  the  proposed  CBE  system  and  to  arrive 
at  a  system  with  a  greater  degree  of  cost  avoidance.  The  section  on  sensi¬ 
tivity  analyses  addresses  these  areas,  which  should  receive  continued 
command  attention. 

The  economic  analysis  presented  in  the  next  section  should  be  considered 
a  model  against  which  future  actual  costs  can  be  compared.  In  addition 
to  the  economic  comparison  of  the  recommended  CBE  system  with  current 
lecture  instruction,  information  was  included  that  points  out  some  of  the 
cost  differences  that  might  occur  if  similar  training  requirements  were 
implemented  with  self-paced  individualized  instruction,  but  without  benefit 
of  the  computer. 

To  point  out  to  MCCES  how  major  system  cost-drivers  can  affect  system 
costs  in  both  positive  and  negative  directions,  we  have  included  three 
sensitivity  analyses.  These  analyses  demonstrate:  (1)  what  can  happen 
to  costs  when  the  percent  reduction  in  training  time  due  to  individuali¬ 
zation  varies,  (2)  how  development  costs  change  when  time  to  develop  in¬ 
structional  materials  varies,  and  (3)  how  student  cost  reductions  can  vary 
when  the  number  of  student  entries  is  altered. 

Implementation  Milestones 

A  list  of  milestones  is  provided  to  describe  the  chronological  com¬ 
pletion  of  major  accomplishments  that  are  a  part  of  the  CBE  system  devel¬ 
opment  effort.  The  milestones  are  listed  in  the  order  they  should  occur, 
along  with  the  appropriate  enabling  task  number  for  referencing  back  to 
the  implementation  plan.  The  milestones  that  are  included  represent  the 
completed  development  of  major  system  components  and  are  of  vital  importance 
to  system  success. 
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Task  No. 


Milestone 


Completion  Date 


2.1.1 

Hire  consulting  systems  analyst 

July  1979 

1.1 

MCCES  training  personnel  participate  in 
evaluation  of  proposed  CBE  system. 

August  1979 

6.1.2 

Complete  training  of  personnel  as  instructional 
developers . 

August  1979 

5.6 

Award  the  system  contract. 

November  1979 

2.1.8 

Complete  design  of  instructional  system. 

December  1979 

5.7 

Complete  vendor  training  of  MCCES  personnel  in 
authoring  language. 

December  1979 

5.7.1 

Complete  vendor  training  of  CBE  system  analyst. 

December  1979 

7.2 

MCCES  receives  CMI  instructor  role  model  from 
contractor . 

January  1980 

5.7.2 

Complete  vendor  training  for  system  operators. 

February  1980 

5.8 

Vendor  installs  initial  equipment:  4  management 
terminals  and  6  student  terminals. 

February  1980 

2.2 

Complete  design  of  computer  software  system. 

March  1980 

4.4 

Complete  installation  of  applications  software 
programs  onto  the  new  computer  system. 

April  1980 

7.3 

Complete  development  of  CMI  instructor  training 
package . 

June  1980 

6.1 

Complete  implementation  of  CCMC  instructional 
materials  onto  the  CAI  system. 

June  1980 

3.5 

Complete  development  of  computer  software  systems. 

July  1980 

7.4 

Complete  training  of  personnel  as  CMI  instructors. 

July  1980 

5.9 

Vendor  installs  equipment:  2  OMR's  and  125 
student  terminals. 

August  1980 

7.5 

Complete  placement  of  trained  CMI  instructors 
into  CCMC  course. 

August  1980 

4.9 

Operate  the  CBE  system  with  the  CCMC  course 
online . 

August  1980 

1.5 

Obtain  reaction  from  training  personnel  about 
initial  system  operation  with  CCMC. 

August  1980 

6.2 

Complete  implementation  of  FDLC  onto  system. 

October  1980 
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Task  No. 


Milestone 


Completion  Date 


3.5.11 

Complete  development  of  CBE  system  cost- 
evaluation  monitoring  subsystem. 

January  1981 

6.3 

Complete  implementation  of 

BEC  onto  the  system 

July  1981 

5.10 

Vendor  installs  balance  of 
and  116  student  terminals. 

equipment:  2  OMR’s 

July  1981 

6.4 

Complete  implementation  of 

FROC  on  system. 

June  1982 

6.5 

Complete  implementation  of 

RFC  on  system. 

July  1983 

6.6 

Complete  implementation  of 

GRRC  on  system. 

June  1984 

6.7 

Complete  implementation  of 
CMI  system  of  CBE. 

courses  onto  the 

August  1984 
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ECONOMIC  ANALYSIS  OF  THE  CBE  SYSTEM  AN1'  CURRENT  LECTURE  INSTRUCTION 


An  economic  analysis  was  conducted  Co  assess  the  cost  effectiveness 
of  the  CBE  system  under  development  by  MCCES  relative  to  the  current  lecture 
method  of  instruction  (referred  to  as  "conventional  instruction"  (Cl)). 

The  criterion  for  identifying  the  cost-effective  method  of  instruction 
was  defined  in  terms  of  least  cost,  assuming  equal  training  effectiveness. 
This  assumption  does  ignore  the  possible  positive  benefits  from  indivi¬ 
dualized  learning  such  as  allowing  for  different  learner  styles  and  abili¬ 
ties.  Life  cycle  cos .  (LCC)  estimates  were  developed  for  the  CBE  system 
configuration. 

This  analysis  differs  from  previous  economic  analyses  that  were  con¬ 
ducted  by  MCCES  personnel  and  revised  by  the  Naval  Training  Equipment 
Center  (NTEC).  Results  of  these  analyses,  which  compared  the  cost  effec¬ 
tiveness  of  specific  CAT  hardware  configurations  to  the  current  lecture 
method  of  instruction,  were  used  to  justify  expansion  of  the  existing  "CAI" 
system.  Another  major  difference  between  the  present  and  previous  studies 
is  the  set  of  courses  identified  for  CBE  implementation.  Specifically, 
earlier  analyses  considered  the  Basic  Electronic/Fundamentals  of  Digital 
Logic  Course  (BEC/FDLC),  the  Radio  Fundamentals  Course  (RFC),  and  the  Tech¬ 
nician  Theory  Course  (TTC);  whereas,  this  analysis  evaluated  the  Communi¬ 
cation  Center  Mass  Course  (CCMC),  BEC/FDLC,  FRDC,  RFC,  and  the  Ground 
Radio  Repair  Course  (GRRC).1 

MCCES  later  revised  the  NTEC  economic  analysis  to  provide  Headquarters, 
Marine  Corps  with  further  information  on  expansion  of  the  CBE  system.2 
Information  was  provided  on  revised  course  schedules,  terminal  requirements, 
system  design,  benefits  and  revised  cost  estimates,  PERT  charts,  and  reduced 
CAI  Instructor  manning.  The  present  analysis  assumed  a  $1.8  million  ADP 
equipment/software  procurement  action  based  on  a  3-year  acquisition  plan 
outlined  in  the  revised  MCCES  economic  analysis. 


!The  original  economic  analysis  was  referred  to  in  Commanding  General, 
Marine  Corps  Base,  Twentynine  Palms,  CA  ltr  C-E/ACAD:GHS:MT/5200,  Enclosure 
(1)  "Economic  Analysis  of  Computer  Assisted  Instruction  (CAI)  at  Marine 
Corps  Communication-Electronics  Schools  MCCES,"  of  24  March  1976.  The  re¬ 
vised  NTEC  economic  analysis  was  forwarded  to  MCCES  in  CO  NTEC  ltr  N-231:wbe 
of  20  July  1976.  MCCES  submitted  the  revised  analysis  to  Commandant  of 
the  Marine  Corps  (Code  OTTS) ,  Headquarters,  U.S.  Marine  Corps  as  enclosure 
(1)  of  Commanding  General,  MCB,  Twentynine  Palms,  CA  ltr  C/E:ACAD:RGH: hg , 
"Updated  Economic  Analysis  for  Expansion  of  Computer  Assisted  Instruction 
for  the  Marine  Corps  Communication-Electronics  Schools  in  accordance  with 
SECNAV  INSTR  7000. 14B,"  of  13  October  1976. 

^Commanding  General,  MCB,  Twentynine  Palms,  CA  ltr  C/E:ATS:RHS:bjm, 
"Economic  Analysis  of  Computer  Assisted  Instruction  (CAI)  at  Marine  Corps 
Communication-Electronics  Schools  MCCES;  revision  of,"  of  13  April  1977. 


Background 


In  a  Cl  mode,  students  complete  training  as  a  group,  progressing  at 
the  same  pace  for  an  established  period  of  time  commonly  referred  to  as 
"course  length."  An  alternative  to  Cl  is  self-paced  individualized  instruc¬ 
tion,  whicli  can  be  delivered  with  or  without  computer  support.  CBE  refers 
to  an  instructional  delivery  system  comprised  of  computer-managed  instruc¬ 
tion  (CMI)  and  computer-assisted  instruction  (CAI).  The  term  "computer- 
managed  instruction"  is  used  for  this  system  to  describe  a  mode  of  instruc¬ 
tion  that  uses  a  computer  to  assign  students  to  instructional  lessons 
presented  away  from  the  computer,  allocate  resources,  administer  and  score 
tests,  assign  remedial  training  as  necessary,  predict  individual  course 
completion  times,  and  maintain  administrative  records  on  student  performance 
and  progress  as  well  as  instructor  records.  In  CAI,  the  student  interacts 
directly  with  the  computer  for  instruction  through  a  computer  terminal. 

The  computer  presents  lessons,  tutors  and  tests  students,  and  performs 
those  functions  associated  with  CMI. 

Although  it  is  generally  accepted  that  course  length  can  be  reduced 
by  CA1/CMI  using  individualized  self-paced  study,  the  contribution  of  the 
computer  component  itself  toward  additional  reductions  beyond  those  attained 
from  self-pacing  without  computer  is  unknown.  Obviously,  self-paced 
instruction  without  computer  support  sacrifices  the  management  benefits 
of  CMI.  An  unresolved  issue  is  whether  or  not  training  requirements  could 
be  satisfied  using  self-paced  instruction  on  a  large  scale  without  computer 
management.  Because  the  examination  of  this  issue  was  beyond  the  scope 
of  this  analysis,  the  present  report  does  not  address  this  issue  directly. 

The  implementation  plan  recommended  by  NAVPERSRANDCEN  includes  devel¬ 
opment  of  an  additional  capability  that  is  designed  to  provide  MCCES  manage¬ 
ment  with  information  on  system  cost  effectiveness.  This  capability  would 
provide  automated  data  storage/retrieval  procedures  that  will  allow  accurate 
costing  cf  all  different  instructional  methods  used  by  MCCES  by  maintaining 
appropriate  records  on  the  same  data  base.  Since  this  capability  does 
not  presently  exist,  information  necessary  for  evaluating  cost  implications 
lor  CUE  and  current  instructional  procedures  was  collected  from  various 
MCCES  offices.  In  some  cases  where  data  were  limited,  the  principles  of 
materiality  (i.e.,  the  time  and  cost  necessary  to  develop  reliable  cost 
estimates  exceeds  the  practical  value  of  the  cost  information  in  comparing 
and  selecting  among  alternatives)  was  invoked.  Cost  elements  were  not 
excluded  where  the  omission  of  cost  information  would  significantly  affect 
LCC  estimates. 

ft  should  be  recognized,  however,  that  a  number  of  confounding  variables 
(e.g.,  student  aptitudes,  trained  personnel  requirements,  and  instructor 
staffing)  may  cause  estimated  and  actual  costs  to  diverge  when  the  CBE 
system  is  operational.  For  example,  potential  reductions  in  student  or 
instructor  costs  may  or  may  not  be  realized  for  a  number  of  reasons.  Under 
CBE  and  individualized  Instruction,  or  self-paced  instruction  without 
computer  support,  it  is  generally  accepted  that  students  complete  training 
faster,  thereby  reducing  costs  incurred  by  personnel  while  in  training 
status.  These  reductions  are  viewed  as  a  training  cost  avoidance,  since 
military  trainees  are  placed  in  a  productive  job  assignment,  although 


on-the-job  training  typically  continues.  Training  status  is  viewed  as 
essentially  nonproductive  because  a  trainee  contributes  little  or  nothing 
to  force  capability.  Reduction  in  course  length  also  creates  an  opportunity 
to  lower  instructor  staffing  requirements.  To  illustrate,  the  computer 
may  create  an  opportunity  to  substitute  lower  ranking  technicians  or  admin¬ 
istrative  personnel  for  instructors  as  well  as  to  reduce  the  number  of 
instructor  billets.  Of  course,  the  computer  does  introduce  new  personnel 
costs  such  as  the  cost  of  systems  analysts  and  computer  programmers.  These 
kinds  of  manpower  tradeoffs  are  difficult  to  quantify  without  detailed 
manpower  studies  that  consider  the  impact  of  shortened  course  length  on 
the  manpower  staffing  requirements  of  both  individualized  instruction  and 
CBE.  Additional  management  benefits  that  accrue  to  CBE  include  a  greater 
capability  to  monitor  training  and  instructional  material  effectiveness, 
and  to  evaluate  effectiveness  in  terms  of  job  performance.  These  benefits 
are  also  difficult  to  quantify.  It  should  be  noted  that  even  now  the  CBE 
section  is  already  providing  instructional  support  through  limited  test 
scoring. 

Economic  Analysis 

The  proposed  MCCES  CBE  system  and  the  current  lecture  method  of  instruc¬ 
tion  are  compared  in  terms  of  life  cycle  costs  (LCC)  for  FY  1980-89.  The 
CBE  LCC  estimate  also  reflects  initial  start-up  costs  for  FY  1979.  Course 
costs  incurred  during  CBE  implementation  are  time-phased  corresponding 
to  dates  set  forth  in  the  implementation  plan. 

The  cost  of  Cl  and  CBE  was  estimated  using  constant  FY  1979  dollars 
and  a  LCC  estimate  was  developed  by  aggregating  research  and  development 
investment,  and  operation  and  support  costs.  The  training  effectiveness 
of  each  alternative,  measured  by  student  achievement  or  on-the-job  per¬ 
formance,  was  assumed  to  be  equal.  Cost  estimates  were  developed  based 
on  information  provided  by  Marine  Corps  personnel  during  a  week-long  data 
collection  visit  to  the  MCCES  command.  A  list  of  technical  and  cost  data 
elements  considered  in  the  analysis  is  contained  in  Appendix  B. 

Annual  cash  flows  for  FY  1980-89  were  discounted  to  present  value  using 
a  10  percent  discount  rate.3  Present  value  differential  cost  (i.e.,  present 
value  Cl  costs  minus  present  value  CBE  costs)  is  the  relevant  measure  of 
least  cost.  This  measure  takes  into  account  the  time  value  of  money. 

Because  the  economic  life  of  each  alternative  is  equal,  calculation  of 
uniform  annual  cost  is  omitted.  The  assumptions  on  which  these  cost  esti¬ 
mates  are  based  are  described  below.  The  reader  should  become  familiar 
with  these  assumptions  since  changes  in  them  would  affect  LCC  estimates. 

Assumptions 

Economic  Life.  The  economic  life  of  Cl  and  CBE  is  assumed  to  be  10 
years.  This  time  period  is  consistent  with  the  economic  life  used  in 


3SECNAV  INSTR  7000. 14B,  "Economic  Analysis  and  Program  Evaluation  for 
Resource  Management,"  18  June  1975. 
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earlier  analyses  and  Marine  Corps'  plans  to  use  the  computer  hardware  and 
software  for  10  years.4  The  economic  life  of  the  CBE  system  starts  in 
FY  1980  when  reductions  in  training  time  and  student  costs  begin  to  accrue. 

Course  Length  Reduction.  This  variable  is  defined  as  the  difference 
between  the  time  required  to  complete  training  for  a  given  course  under 
Cl  and  the  average  time  students  would  take  to  complete  this  training  under 
CBE.  Variable  course  length  factors  of  20,  32,  and  44  percent  were  applied 
to  the  Cl  course  length  for  each  of  the  six  courses  to  arrive  at  average 
CBE  course  completion  times.  The  most  reasonable  estimate  is  32  percent, 
which  represents  an  average  course  length  reduction  found  in  a  comparison 
of  several  CAI/CMI  studies.5  This  estimate  was  used  in  the  economic 
analysis.  The  20  percent  and  44  percent  figures  were  used  in  a  sensitivity 
analysis. 

Man-hour  Conversion  Factors.  The  amount  of  time  required  to  imple¬ 
ment  CBE  is  assumed  to  vary  with  the  amount  of  CAI  and  CMI  capabilities. 
Program  of  Instruction  (POI)  hours  were  identified  as  either  knowledge- 
related  or  performance-related.  According  to  the  CBE  implementation  plan, 
conversion  of  knowledge-related  POI  hours  will  be  divided  equally  among 
CAI  and  CMI  functions,  while  all  performance-related  POI  hours  will  be 
converted  to  a  CMI  function.  Long,  moderate,  and  high  conversion  factors 
were  used  to  provide  a  reasonable  range  of  instructional  materials  develop¬ 
ment  costs.  Man-hour  conversion  factors  were  specified  as  shown  in  Table 
1.  The  moderate  man-hour  conversion  flow  was  used  in  the  economic  analysis 
of  CBE  and  Cl.  The  higher  and  lower  values,  viewed  as  contingency  assump¬ 
tions,  were  used  in  the  sensitivity  analysis  to  determine  how  instructional 
development  costs  might  vary. 


Table  1 

Estimated  Hours  for  Material  Development 
Per  Hour  of  CAI  or  CMI  Instruction 


Low 

Moderate 

High 

Mode 

Author 

Code 

Author 

Code 

Author 

Code 

CAI 

75 

25 

100 

50 

150 

75 

CMI 

75 

10 

100 

20 

150 

30 

4Director  of  Training,  Headquarters,  U.S.  Marine  Corps  (Code  OTTB), 

"Front  End  Analysis  to  Develop  an  Implementation  Plan  for  the  Computer 
Based  Education  (CBE)  System  at  the  Marine  Corps  Communication-Electronics 
Schools  (MCCES),"  13  February  1979. 

50rlansky,  J. ,  &  String,  J.  Cost-effectiveness  of  computer-based  instruc¬ 
tion  in  military  training  (10A  Paper  N  P-1375).  Arlington,  VA:  Institute 
for  Defense  Analysis,  April  1979. 
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Man-year  Conversion  Costs.  A  representative  pay  grade  of  E-6  was  used 
for  costing  military  personnel  services  associated  with  CBE  instructional 
materials  development .  The  composite  standard  military  rate  for  E-6  Marine 
Corps  personnel  Is  $12, 704. 6  This  rate  does  not  reflect  military  retire¬ 
ment,  leave,  holiday,  and  other  personnel  overhead  costs.  The  E-6  cost 
factor  approximates  the  average  cost  of  military  and  civilian  instructors 
($12,800)  presently  assigned  to  the  six  courses  selected  for  CBE  implemen¬ 
tation.  Direct  man-years  required  to  author  and  code  CBE  instructional 
materials  were  costed  using  the  E-6  cost  factor.  Man-year  requirements 
were  computed  by  dividing  estimated  course  development  man-hours  by  the 
number  of  direct  man-hours  in  a  man-year.  The  number  of  direct  military 
man-hours  per  man-year  is  about  1693  hours.  This  estimate  is  based  on 
52  40-hour  work  weeks  less  time  off  for  leave  (173  hours),  holidays  (72 
hours),  rifle  range  (72  hours),  and  miscellaneous  duties  (104  hours). 

Direct  man-hours  in  a  civilian  contractor  man-hour  is  assumed  to  equal 
1920  hours  (i.e.,  48  40-hour  work  weeks). 

Personnel  Costs.  Personnel  costs  are  comprised  of  military  and 
civilian  personnel  costs.  Composite  standard  military  rates  for  Marine 
Corps  personnel  were  used  to  estimate  military  instructor  and  student  costs. 
Instructor  costs  were  computed  using  the  E-6  composite  standard  military 
rate;  and  student  costs,  by  multiplying  the  number  of  students  per  pay 
grade  by  the  appropriate  composite  standard  military  rate.  Percentage 
distributions  of  students  by  pay  grade  were  derived  for  each  course  using 
sample  data  extracted  from  rank  breakdown  reports  for  October  1978  through 
February  1979.  The  assumption  was  made  that  these  distributions  would 
remain  constant  for  FYs  1980-89.  In  addition,  these  distributions  were 
extrapolated  to  CBE  courses.  Civilian  personnel  services  were  costed  using 
general  schedule  rates,  assuming  each  grade  had  a  step-5  level  on  the 
average.  The  general  schedule  rate  was  accelerated  by  29  percent  to  account 
for  leave  and  benefits  such  as  retirement  and  insurance. 

Student  Entries.  The  number  of  students  entering  training  in  a  given 
fiscal  year  fluctuates  for  various  reasons  such  as  trained  personnel  re¬ 
quirements  and  attrition  rates.  High  and  low  student  flow  scenarios  were 
developed  for  each  course  to  allow  for  variable  student  entry  conditions. 
These  scenarios  are  based  on  FYs  1977-80  MCCES  formal  school  schedules 
which  list  total  quotas,  or  student  entries,  for  MCCES  courses.  A  high 
student  flow  condition  in  any  1  year  would  be  about  5000  trainees;  and 
a  low  flow  condition,  about  4200  students.  The  assumption  was  made  that 
either  a  high  or  low  flow  condition  would  prevail  throughout  FYs  1980-89. 
This  approach  assumes  that  all  students  entering  training  would  graduate; 
that  is,  zero  attrition.  Incomplete  attrition  data  for  several  Cl  courses 
and  no  attrition  data  for  CBE  courses  did  not  permit  an  evaluation  of 
attrition  costs.  As  a  result,  student  cost  estimates  are  slightly  over¬ 
stated.  This  is  inconsequential  since  attrition  costs  probably  would  be 
almost  equivalent  under  CBE  or  Cl.  Annual  and  life  cycle  student  costs 
are  detailed  in  Appendix  C. 


6NAC0MPT  NOTE  7041,  "Composite  Standard  Military  Rates,"  6  December 
1978. 


Omission  of  Sunk  Costs  and  Common  Costs.  Sunk  costs  (i.e.,  costs 
incurred  as  a  result  of  irrevocable  decisions)  and  common  costs  (i.e., 
costs  incurred  to  support  two  or  more  activities  or  alternatives)  were 
omitted  from  the  analysis.  Since  Cl  is  fully  operational,  past  expendi¬ 
tures  for  research  and  development  and  investment  were  treated  as  sunk 
costs.  Furthermore,  no  future  research  and  development  nor  investment 
costs  are  planned  for  FYs  1979-89  under  the  current  lecture  method  of 
instruction.  Costs  considered  as  common  costs  included:  training  overhead 
costs,  base  support  costs,  and  temporary  additional  duty  costs.  To  obtain 
an  idea  of  other  excluded  costs,  refer  to  Appendix  B. 

Cost-estimating  Equations 

LCC  is  composed  of  three  cost  categories:  research  and  development, 
investment,  and  operation  and  support.  Component  cost  estimates  were 
derived  using  appropriate  equations  that  represent  functional  relationships 
between  the  dependent  variable  (cost)  and  independent  variables.  Table 
2  presents  equations  used  in  determining  fiscal  year  costs  for  each  cost 
element.  Component  LCC  estimates  were  obtained  by  aggregating  projected 
fiscal  year  costs  for  each  cost  element  across  the  six  courses. 

System  Cost  Estimates 

The  analysis  of  Cl  and  CBE  LCCs  was  based  on  several  key  assumptions 
concerning  course  length  savings,  man-hour  conversion  factors,  and  student 
flow  scenarios.  Different  values  were  specified  in  order  to  measure  the 
sensitivity  of  CBE  cost  projections  to  changes  in  expected  course  length 
reductions,  man-hour  conversion  requirements,  and  the  number  of  students 
entering  training.  The  economic  analysis  assumes:  (1)  an  average  32 
percent  reduction  in  CBE  training  time  compared  to  Cl  course  length, 

(2)  150  man-hours  for  conversion  of  a  POI  hour  to  CAI  and  120  man-hours 
for  conversion  of  a  POI  hour  to  CMI,  and  (3)  a  high  annual  student  flow 
of  about  5000  trainees.  LCC  projections  developed  using  these  values 
provide  the  most  reasonable  estimates  for  the  comparing  costs  of  CBE  and 
the  current  lecture  method. 

Results  of  this  discounted  LCC  analysis  indicate  that  CBE  is  the  pre¬ 
ferred  alternative  based  on  the  least-cost  criterion,  as  shown  in  Table 
3.  The  estimated  present  value  differential  costs  associated  with  the 
recommended  CBE  system  configuration  is  about  $5.5  million.  This  finding 
means  that  CBE  may  be  cost  effective  only  if  the  original  assumptions  are 
met  and  only  when  compared  to  the  current  alternative  Cl. 

Nondiscounted  costs  of  CBE  and  Cl  are  presented  in  Table  4.  In  non- 
discounted  terms,  CBE  results  in  a  training  cost  avoidance  of  approximately 
$12.2  million.  Approximately  92.4  percent  of  the  LCCs  traceable  to  CBE 
(i.e.,  excluding  the  Cl  cost  share  incurred  during  CBE  implementation) 
are  accounted  for  by  recurring  annual  operation  and  support  costs.  CBE 
research  and  development  and  investment  costs  account  for  about  0.4  and 
7.2  percent  of  nondiscounted  LCCs  respectively. 
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Table  2 


Cost  Estim.it  ing  Equations  by  Cost  Element 


Cost  Element  Equations 


Course  Development 


CD  -  [ 
where : 


[  ,5K  (CAI  ±  CM  I)  +  (?)  (CMI)1 
MH 


x  AMC 


+  MC 


CD 

ZK 

Z? 

CAI 

CMI 

MH 

AMC 

Mci 

“  CBE  course  development  costs 

■  Number  of  knowledge-related  P01  hours  under  Cl  for  each 
course 

■  Number  of  performance-related  POI  hours  under  Cl  for 
each  course 

*  CAI  man-hour  course  conversion  factor 

■  CMI  man-hour  course  conversion  factor 

*  Number  of  direct  man-hours  per  man-year 

■  Average  E-6  man-year  cost 

*  Material  costs  per  year 

CMI  Test  Coding 

T 

-  MY  x  AMC 

where: 

T 

*  Cost  of  coding  course  tests  on  computer 

MY 

*  Estimated  man-year  to  code  tests  on  computer 

AMC 

*  Average  E-6  man-year  cost 

Initial  Instructor  Training 

IT 

-  ID  +  (I  x  CL  x  MW) 

whe  re : 

IT 

*  Initial  CBE  instructor  training  costs 

i 

»  Number  of  CBE  instructors 

ID 

*  CBE  instructor  training  initial  course  development 

ICL 

“  Course  length  of  initial  training  in  weeks 

MW 

•  Average  cost  per  E-6  instructor  man-week 

CBE  Section 

CBES 

-  MP  +  CP  +  PS  +  S 

whe  re : 

CBES 

“  CBE  section  costs 

MP 

*  Cost  of  military  personnel  services 

CP 

«  Cost  of  civilian  personnel  services 

PS 

■  Budget  cost  of  purchased  services  including  computer 

maintenance  and  other  miscellaneous  costs 

s 

»  Budget  cost  of  supplies/consumables 

Instructor  Costs 

IC 

-  I  x  AIC 

where: 

IC 

■  Instructor  costs 

I 

-  Number  of  instructors  assigned 

AIC 

■  Average  instructor  man-year  cost 

Recurring  Instructor  Training 

RT 

-  (i^)  x  CL  x  MW 

where: 

RT 

•  Cost  of  recurring  instructor  training 

1 

»  Number  of  instructors 

LA 

-  Average  length  of  instructor  assignment  in  years 

CL 

-  Course  length  of  instructor  training  course 

MW 

»  Average  cost  per  instructor  man-week 

Student  Personnel  Costs 

SC 

■  S  *  IPG  s  CL  x  MPC 

where: 

sc 

•  Cost  of  military  personnel  in  training 

s 

■  Number  of  military  personnel  for  high/low  training 

scenarios 

XPC 

-  Percentage  distribution  of  personnel  for  training 

status  (by  pay  grade) 

CL 

*  Course  length  in  weeks  by  method  of  instruction 

MPC 

■  Weekly  military  personnel  cost  by  rank 
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Discounted  Life  Cycle  Costs  of  the  Computer-based  Education  (CBE) 
and  Conventional  Instruction  (Cl) 


comparisons 


A  measure  of  payoff  was  developed  to  compare  the  cost  avoidance  as¬ 
sociated  with  reduced  training  time  on  a  course-by-course  basis.  This 
measure  is  a  ratio  of  differential  student  costs  (Cl  student  costs  minus 
CBE  student  costs,  where  CBE  student  costs  equal  CBF.  student  personnel 
costs  plus  the  Cl  cost  share  incurred  during  system  conversion)  to  course¬ 
ware  development  costs.  These  ratios  used  to  aid  in  selecting  courses 
for  implementation  are  displayed  in  Table  5.  In  general,  these  ratios 
would  change  given  a  different  implementation  period. 


Table  5 

Differential  Student  Costs  to  Instructional 
Development  Cost  Ratios  by  Course,  FY’s  1980-89 


Course 

Differential  Student 
Cost,  FY's  1980-89 

Course  Development'5 
Cost,  FY’s  1980-89 

Cost  Ratio 

CCMC 

$2,993,592 

$313,292 

9.6 

FDLC 

1,191,213 

67,839 

17.6 

BEC 

5,459,734 

412,653 

12.9 

FROC 

3,818,804 

264,302 

14.4 

RFC 

881,575 

218,136 

4.0 

CRRC 

2,154,976 

686,213 

3.1 

Based 

on  high  student  flow  and  CBE 

course  length  reduction  of 

32  percent. 

Based  on  moderate  man-hour  conversion  factors,  assuming  military  developers. 


Additional  supporting  information  used  to  derive  life  cycle  costs 
and  conduct  the  study  is  presented  in  Appendix  C.  Types  of  information 
provided  in  Appendix  C  include:  (1)  C3E  conversion  hours  for  courses, 
course  implementation  sequence,  detailed  yearly  student  costs  for  low 
and  high  scenarios,  (2)  estimated  costs  for  the  extant  CBE  section  at 
MCCES,  (3)  course  development  materials  costs,  and  (4)  student  costs 
incurred  during  conversion  to  CBE. 

Other  Cost  Considerations 


A  number  of  cost  considerations  are  mentioned,  although  detailed  review 
of  them  is  beyonu  the  scope  of  this  analysis.  These  factors  are  briefly 
discussed  to  provide  information  that  Marine  Corps  personnel  should  con¬ 
sider  in  developing  the  CBE  system. 

1.  Instructor  manning  under  CBE  will  be  less  than  staffing  require¬ 
ments  for  lecture  instruction.  What  is  sometimes  overlooked,  however, 
is  that  instructor  reductions  are  offset  by  new  requirements  for  computer 
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support  personnel.  A  manpower  staffing  study  is  needed  to  determine  actual 
CBE  instructor  and  computer  support  personnel  requirements.  This  study 
should  be  conducted  after  the  computer  system  has  been  selected  in  order 
to  make  more  accurate  manpower  estimates.  If  sclf-paced  instruction 
without  computer  support  was  used  instead  of  CBE,  the  number  of  instructors 
required  would  probably  be  no  less  than  is  needed  for  group  lecture  instruc¬ 
tion  given  current  MCCES  manning.  In  fact,  individualized  instruction, 
on  the  proposed  scale,  if  it  were  not  supported  by  computer  capabilities, 
could  possibly  require  more  personnel  to  handle  the  added  duties  associated 
with  individualized  study  and  management.  A  manually  managed  individualized 
self-paced  system  would  not  require  an  investment  in  ADP  equipment/software 
nor  outlays  for  its  operation  and  maintenance,  although  other  equipments 
(e.g.,  optical  mark  readers)  might  be  required.  In  addition,  a  significant 
Increase  in  administrative  support  personnel  would  be  required  to  develop 
manually  the  1700-plus  individualized  learning  prescriptions,  including 
the  accounting  and  allocation  of  training  resources  to  maximize  learning 
efficiency  with  minimum  queing  delays. 

2.  Instructor  training  must  be  conducted  regardless  of  whether  the 
instruction  is  provided  by  lecture  or  by  CBE;  instructor  training  costs 
will  be  increased,  however,  during  the  period  that  the  CBE  system  is  being 
installed  because  dual  training  will  be  required.  That  is,  during  the 
conversion  period,  training  will  be  required  for  both  lecture  and  CBE 
instructors. 

3.  Although  student  personnel  costs  are  expected  to  decrease  with 
CBE,  this  reduction  will  occur  only  if  full  CBE  capabilities  are  developed 
and  used.  Particular  attention  needs  to  be  directed  toward  ensuring  that 
students  completing  a  CBE  course  go  directly  into  another  course  or  to 
their  job  assignment.  To  the  extent  that  students  remain  in  a  waiting 
status,  student  costs  will  increase  with  no  obtained  benefit.  Detailed 
estimated  student  costs  for  the  CBE  system  are  presented  in  Appendix  C. 

4.  Instructional  materials  development  costs  are  high  for  any  form 
of  individualized  instruction.  Costs  for  developing  materials  for  a  CMI, 
especially  for  CAI  systems,  are  even  higher  because  of  the  need  to  code 
the  materials  onto  the  computer.  Table  6  presents  the  estimated  total 
materials  development  costs  for  the  CBE  system,  divided  into  authoring 
and  coding  categories.  For  comparison,  these  costs  for  a  noncomputer- 
based  individualized  system,  which  needs  only  materials  authoring,  are 
also  included. 

5.  To  resolve  the  question  of  the  relative  contribution  of  the 
computer  to  reducing  individualized  instruction  training  time,  attention 
should  be  directed  toward  comparing  the  effects  of  a  CMI/CAI  course  managed 
both  with  and  without  a  computer.  This  comparison  would  provide  signi¬ 
ficant  information  regarding  the  relative  rates  of  course  completion 
through  a  course  that  is  operated  both  with  and  without  computer  support. 
This  information  could  be  used  to  modify  future  implementation  plans  as 
required.  During  this  comparison,  all  efforts  should  be  made  to  determine 
not  only  the  costs  of  system  operation  but  also  the  benefits  provided 

to  management  and,  of  course,  to  the  student. 


Table  6 


Instructional  Development  Costs 


Task 

CBE  ($K) 

Individualized  ($K) 

Authoring 

1490 

1490 

Coding 

413 

0 

Total 

1903 

1490 

In  the  following  section,  two  cost  sensitivity  analyses  are  presented. 
These  analyses  examine  the  effects  on  the  system  caused  by  varying  the 
percent  reduction  in  CBE  course  completion  times,  student  entries,  and 
estimates  of  the  necessary  time  to  develop  CBE  instructional  materials. 


COST  SENSITIVITY  ANALYSES 


Sensitivity  analyses  were  performed  to  examine  the  variation  of  life 
cycle  costs  (LCC)  estimates  resulting  from  changes  in  the  assumed  values 
of  CBE  course  length  reductions,  annual  student  flow  conditions,  and  CBE 
man-hour  conversion  factors.  As  previously  noted,  contingency  assumptions 
were  formulated  to  assess  the  cost  effectiveness  of  CBE  if  values  for 
these  three  variables  vary. 

These  analyses  suggest  that  CBE  system  cost  effectiveness  is  insensi¬ 
tive  to  contingency  values  presented  above  and  assumes  military  developers. 
The  greatest  degree  of  cost  uncertainty  involves  estimates  of  CAI  and 
CMI  instructional  development  times  and  costs.  Increases  in  actual  CAI/CMI 
development  times  could  lengthen  the  period  of  implementation  and  result 
in  higher  student  costs,  as  well  as  higher  instructional  development  costs. 
Conversely,  decreased  CAI/CMI  development  time  would  result  in  earlier 
implementation  and  lower  student  and  development  costs.  If  the  implemen¬ 
tation  period  was  prolonged  by  more  than  2  years,  the  CBE  system  could 
become  more  costly  than  Cl.  For  this  reason,  MCCES  should  continually 
monitor  actual  man-hours  expended  to  write  and  code  lesson  materials. 

Nondiscounted  LCC  estimates  based  on  the  economic  analysis,  including 
the  sensitivity  analyses,  are  summarized  in  Table  7.  Separate  comparisons 
are  made  for  high  and  low  student  flow  scenarios  since  the  number  of 
student  entries  is  independent  of  the  method  of  instruction.  Estimates 
for  Cl  vary  only  as  a  function  of  student  entries.  CBE  estimates  vary 
as  a  function  of  student  entries,  course  length  reductions,  and  CBE  in¬ 
structional  development  times.  The  latter  are  based  on  military  instruc¬ 
tional  developers. 


Table  7 

LCC  Comparisons  by  Student 
Flow  and  Method  of  Instruction 
(In  Millions  of  Dollars) 


High  Student 

Flow 

Low 

Student  Flow 

Outcome 

CBE 

Cl 

CBE 

Cl 

Optimistic 

63.9 

82.9 

57.8 

73.9 

Most  Likely 

70.7 

82.9 

63.8 

73.9 

Pessimistic 

77.9 

82.9 

70.2 

73.9 

Several 

observations  concerning 

this  table  are 

in  order. 

First,  a 

high  student  entry  scenario  is  expected  to  prevail  over  the  10-year  period. 
Thus,  more  confidence  should  be  placed  on  LCC  estimates  based  on  the  high 
flow  case.  Second,  CBE  costs  could  increase  to  the  extent  that  CBE  instr-i'- 
tional  materials  are  developed  under  contract.  The  "best  estimate"  of 
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contract  cost  is  nearly  $3.4  million,  which  covers  instructional  materials 
development  for  all  six  courses.  In  the  event  of  low  personnel  requirements, 
and  an  average  course  length  reduction  of  20  percent  under  CBE,  there 
is  a  low  risk  that  CBE  and  Cl  would  cost  about  the  same,  assuming  that 
all  instructional  materials  were  developed  under  contract. 

Reductions  in  Training  Time 

It  is  generally  recognized  that  computer-based  training,  with  its 
self-paced  elements,  results  in  significantly  less  student  training  time 
than  conventional  lecture  instruction.  Actual  reductions  in  training 
time  have  varied  from  12  to  89  percent.  When  analyzing  a  proposed  CBE 
system,  it  is  necessary  to  select  a  percent  figure  for  computing  student 
training  costs.  Although  estimates  vary  widely,  MCCES  selected  a  35  per¬ 
cent  reduction  in  their  economic  analysis  supporting  the  CBE  system  ac¬ 
quisition.  A  slightly  more  conservative  figure  of  32  percent  was  chosen 
for  the  present  analysis.  This  figure  was  the  average  training  time 
reduction  found  in  Orlansky  and  String's7  comparison  of  several  CAI/CMI 
studies  (1979). 

A  cost  sensitivity  analysis  was  performed  to  determine  how  student 
costs  would  be  affected  by  higher  or  lower  percentage  reduction  in  course 
length  under  CBE.  Forty-four  percent  as  a  high  estimate  was  selected 
because  that  was  the  median  percent  savings  reported  by  Orlansky  and  String 
when  they  examined  direct  CMI  and  lecture  instruction  comparisons. 

Twenty  percent  was  chosen  as  the  low  estimate  since  it  was  equally  distant 
from  32  percent,  but  in  a  conservative  direction.  These  reductions, 
expressed  as  a  percent  of  Cl  course  length,  translate  into  CBE  training 
times  of  80  percent,  68  percent,  and  56  percent. 

The  sensitivity  of  CBE  and  Cl  student  costs  was  also  investigated 
based  on  variable  student  entry  conditions  of  either  4997  or  4180  trainees 
per  year.  Reductions  in  training  time  have  a  considerable  effect  on 
personnel  costs,  and  this  effect  increases  (decreases)  as  more  (fewer) 
students  undergo  training.  Table  8  illustrates  how  student  costs  vary 
as  a  function  of  different  training  times  and  student  entry  levels. 

The  cost  avoidance  associated  with  the  individualized  nature  of  CBE 
would  increase  substantially  in  the  event  of  a  full  mobilization  of  the 
Marine  Corps  and  concomitant  rise  in  training  loads.  It  is  unlikely, 
however,  that  the  improved  training  efficiency  could  be  sustained  under 
individualized  instruction  without  computer  management.  Under  conditions 
of  relatively  high  student  input,  there  would  be  a  normal  cadre  of  CBE 
system  instructors.  The  question  remains  as  to  how  the  system  would  adjust 
to  a  surge  in  daily  student  loads  from  approximately  1700-1900  students 
to  as  many  as  5000  students  daily.  The  answer  requires  increasing  the 
number  of  (1)  students  in  a  learning  center,  (2)  operating  learning  centers, 
and  (3)  shifts  that  are  operated  each  day.  In  the  Navy  CMI  system,  this 
ability  to  adjust  to  varying  loads  has  been  demonstrated  during  a  period 
of  high  student  input  when  one  of  the  Basic  Electricity  and  Electronics 
Schools  operated  three  full  6-hour  shifts  per  day. 


7  See  footnote  5. 


22 


Cost  Sensitivity  Analysis:  Variable  Student  Flow  Scenarios 
by  Method  of  Instruction  and  CBE  Course  Length  Reduction 


r  Line  to  Develop  Instructional  Materials 


I  he  amount  of  effort  necessary  to  develop  individualized  materials 
for  computer-based  training  is  enormous.  It  is  only  through  expenditure 
of  this  effort,  though,  that  the  benefits  of  individualized  instruction 
c.in  be  obtained.  Because  of  the  large  amount  of  time  and  money  needed 
for  development,  this  time  factor  must  be  monitored  closely.  In  the  early 
years  of  CAI,  some  estimates  of  the  amount  of  time  necessary  to  develop 
1  hour  of  instructional  material  were  as  high  as  600-700  hours.  As  CAI 
technology  and  the  ISD  process  have  developed,  the  estimates  and  the  actual 
times  have  dropped  considerably.  CBE  instructional  development  and  coding 
times  are,  however,  still  subject  to  a  great  deal  of  variability,  and  so 
the  costs  can  vary  considerably. 

For  the  recommended  CBE  system,  a  moderate  time  for  instructional 
material  authoring  was  selected — 100  hours  per  hour  of  instruction.  The 
time  required  for  coding  the  instructional  and  test  material  onto  the  system 
was  estimated  at  20  hours  per  CHI  hour,  and  50  hours  per  CAI  hour  of  instruc¬ 
tion.  For  the  sensitivity  analysis,  higher  and  lower  figures  were  devel¬ 
oped.  These  figures,  presented  previously  in  Table  1,  were  chosen  because 
they  represent  realistic  estimates  of  what  could  occur  if  instructional 
development  time  is  not  closely  monitored. 

Authoring,  moderately  estimated  at  100  hours  per  instructional  hour, 
includes  those  activities  necessary  to  conduct  a  brief  analysis  of  existing 
training  materials,  development  of  training  objectives  and  measurement 
standards,  preparation  of  lesson  and  test  materials,  and  the  graphic  support 
work  to  produce  illustrated  material.  An  extensive  requirement  for  multiple 
media  formats  is  not  included  in  these  estimates.  Coding  time,  moderately 
estimated  at  20  to  50  hours  per  CHI  or  CAI  hour,  includes  these  activities 
required  to  properly  code  developed  instructional  materials  into  the  CMI 
or  CAI  system.  These  estimates  were  applied  to  both  military  and  civilian 
developers.  The  lower  conversion  values  represent  optimal  times  (costs) 
for  contractor  personnel  with  broad  experience  in  CAI/CMI  instructional 
materials  development.  The  high  conversion  factors  represent  figures 
consistent  with  some  actual  materials  development  experiences.  The  moderate 
value  represents  the  "best  guess"  of  current  development  organizations. 

Results  of  the  instructional  man-year  requirements  cost  sensitivity 
analyses  are  displayed  in  Table  9.  Instructional  development  costs  com¬ 
prise  the  largest  outlay  in  CBE  development.  The  amount  of  time  expended 
to  develop  instructional  materials  and  implement  the  CBE  system  requires 
careful  monitoring,  since  optimal  training  efficiency  depends  on  timely 
implementation.  Prolonged  implementation  will  vitiate  student  cost  re¬ 
ductions  associated  with  CBE. 

Even  though  it  would  be  less  costly  to  use  military  personnel  for 
instructional  material  development,  the  material  developed  by  the  civilian 
professional  developer  would  surely  be  better.  Better  instructional 
material  would  result  in  better  student  achievement,  perhaps  even  less 
training  time.  Excellent  instructional  materials  in  the  Marine  Corps 
schools  might  indirectly  improve  recruitment  incentive  if  trainees  report 
favorably  to  their  friends  about  the  training  experience.  Admittedly, 
there  is  no  objective  evidence  to  support  this  projected  benefit. 
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Cost  Sensitivity  Analysis:  Man-year  Conversion  Factors 
bv  Course  and  Type  of  Man-year 


ii 

■j  Course 

Military 

Contract 

1 

Low 

Moderate 

High 

Low 

Moderate 

High  1 

ij  CBE  Man-year  Course  fl 

(  Development  Requirements  (Man-years)  uj 

!  ; 

■:  CCMC 

17.13 

24.43 

36.64 

15.16 

21.62 

32.43  \ 

FDLC 

3.68 

5.29 

7.94 

3.25 

4.68 

7.03  \ 

t.  BEC 

19.10 

27.59 

41.38 

16.90 

24.42 

29.70  ! 

FROC 

14.42 

20.61 

30.89 

12.76 

18.23 

27.34 

RFC 

11.79 

17.01 

29.51 

10.44 

15.05 

26.12  { 

GRRC 

37.15 

53.51 

80.27 

32.87 

47.36 

71.03 

9 

Total 

103.27 

148.44 

226.63 

91.38 

131.36 

193.65 

!! 

CBF.  Man-year  Course  Development  Costs 

($) 

.  ... 

CCMC 

217,619 

310,359 

465,475 

586,500 

810,750 

1,216,125 

•’  FDLC 

46,751 

67,204 

100,870 

121,875 

175,500 

263,625 

'■  BEC 

242,646 

350,503 

525,691 

633,750 

915,750 

1,113,750 

1  FROC 

183,192 

261,829 

392,427 

478,500 

683,625 

1,025,250 

l  RFC 

149,780 

216,095 

374,895 

391,500 

564,375 

979,500 

I  GRRC 

471,954 

679,792 

1,019,750 

1,232,625 

1,776,000 

2,663,625 

I  Total 

1,311,942 

1,885,782 

2,879,108 

3,426,750 

4,926,000 

7,261,875 

Notes. 

1.  Different  man-hour  conversion  factors  for  CMI  and  CAI  were  applied  to  course  hours 
as  follows:  CMI  conversion  factors  used  for  all  performance-related  POI  hours 
and  50  percent  of  knowledge-related  POI  hours;  CAI  conversion  factors  were  applied 
to  remaining  knowledge-related  POI  hours.  Direct  military  man-hours  in  a  man-year 
estimated  at  1693. 

2.  Low  man-hour  conversion  factors  for  CMI  and  CAI  were  specified  as  follows: 

CMI — 75  man-hours  for  authoring  and  10  mar.-hours  for  coding;  CAI — 75  man-hours 
for  authoring  and  25  man-hours  for  coding. 

3.  Medium  man-hour  conversion  factors  for  CMI  and  CAI  were  specified  as  follows: 

CMI — 100  man-hours  for  authoring  and  20  man-hours  for  coding;  CAI — 100  man¬ 
hours  for  authoring  and  50  man-hours  for  coding. 

4.  High  man-hour  conversion  factors  for  CMI  and  CAI  were  specified  as  follows: 

CMI — 150  man-hours  for  authoring  and  30  man-hours  for  coding;  CAI — 150  man¬ 
hours  for  authoring  and  75  man-hours  for  coding. 


5.  Military  man-years  were  estimated  at  $12,704  (E-6)  while  contract  man-years  were 
estimated  at  $37,500. 


Since  costs  are  so  high  for  civilian  development,  an  alternative 
would  be  to  have  civilians  develop  the  first  two  courses,  one  a  main¬ 
tenance  course  (FDLC)  and  the  other,  an  equipment  operating  course  (CCMC). 
Those  course  materials  could  then  serve  as  models  for  military  instruc¬ 
tional  developers  working  on  subsequent  courses.  This  approach  would 
require,  using  the  moderate  conversion  rates  for  development,  approxi¬ 
mately  $175K  and  $811K  for  FDLC  and  CCMC  respectively. 


RESEARCH  AND  DEVELOPMENT  RECOf  1MEND AT  IONS 


The  research  and  development  recommendations  presented  in  this  section 
include  both  those  developmental  ei forts  that  are  necessary  for  successful 
CBE  system  design  and  development,  and  the  research  efforts  that  will 
serve  to  make  ihe  MCCES  system  fully  functional  and  place  it  at  the 
forefront  of  large-scale,  operational,  computer-based  training.  If 
developed  according  to  the  recommended  implementation  plan,  the  MCCES 
system  will  offer  the  Marine  Corps  the  opportunity  for  a  training  system 
that  goes  beyond  the  capabilities  of  any  existing  computer-based  system. 
Such  a  system  would  noi  only  be  able  to  manage  and  monitor  the  progress 
of  individual  students  through  individual  technical  courses,  but  also 
through  a  sequence  of  courses.  Additionally,  the  system  would  be  able 
to  relate  training  achievement  to  actual  job  performance  through  an 
external  evaluation  system,  and  to  provide  total  internal  evaluation 
of  instructional  materials  and  system  cost  effectiveness. 

Research  Efforts 


To  realize  the  full  potential  of  this  planned  CBE  system,  the  four 
recommended  research  efforts  discussed  in  the  following  pages  must  be 
accomplished.  The  first  two  projects — system  analyses  and  CM I  instruc¬ 
tor  training — are  necessary  for  successful  initial  system  design  and 
implementation;  and  the  second  two — time  management  and  system  evalua¬ 
tion — for  developing  the  full  system  capability.  Each  of  these  efforts 
will  be  described  separately. 

System  Analysis 

Although  this  effort  is  Included  In  the  R&D  section,  it  is  a  vital 
component  of  the  implementation  plan  and  must  be  accomplished  to  achieve 
a  working  CBE  system.  The  purpose  of  this  work  is  to  conduct  the  system 
analysis  tasks  that  will  provide  the  general  framework  and  the  detailed 
computer  software  for  the  entire  CBE  system.  This  framework  will  give 
organizational  structure  to  the  computer  processes  that  are  required 
to  satisfy  all  the  functional  requirements.  In  general,  this  involves 
conducting  the  analysis  required  to  lay  out  and  document  the  computer 
processes  and  file  structures  that  will  enable  the  system  to  provide 
the  desired  instructional,  managerial,  and  evaluation  capabilities  when 
fully  implemented. 

This  task  is  absolutely  vital  to  system  success  and  must  be  ac¬ 
complished  by  a  highly  trained  systems  analyst,  ideally,  one  experienced 
with  large-scale  military  CMI  systems.  This  expertise  does  not  presently 
exist  within  MCCES  personnel  resources.  To  obtain  the  level  of  product 
necessary,  a  contracted  or  consultant  effort  would  probably  be  required 
to  accomplish  this  work.  Suitable  contractors  or  consulting  systems 
analysts  (CSA)  should  have  extensive  computer  systems  analysis  experience, 
familiarity  with  computer-based  training,  and  intimate  experience  with 
large  scale  military  technical  training. 

The  CSA  would  be  charged  with  developing  the  total  CBE  system  design 
and  would  work  with  the  analysts  provided  by  the  vendor  and  the  MCCES 
to  assure  compatibility  of  functional  system  requirements  with  machine 


processing  capabilities.  The  CSA  should  work  from  the  CBE  office  and  report 
to  the  military  CBE  project  officer.  Systems  analyst  personnel  now  existing 
and  planned  for  the  CBE  office  should  be  able  to  provide  sufficient  support 
to  the  CSA  to  accomplish  actual  system  programming. 

The  CSA  should  be  working  at  this  time.  Ten  to  twelve  months  of  effort 
are  probably  required  and  it  is  estimated  that  this  will  cost  approximately 
$50,000  during  FY  1980.  Detailed  information  regarding  the  actual  tasks 
to  be  accomplished  by  the  CSA  are  provided  in  the  implementation  plan  under 
Functional  Objectives  2.0  (Design  CBE  System)  and  3.0  (Develop  Computer 
Software  Programs). 

CHI  Instructor  Training 

The  second  effort  that  must  be  accomplished  to  ensure  successful  initial 
system  implementation  is  the  development  of  a  CMI  instructor  training  pack¬ 
age.  Current  training  of  lecture  instructors  is  not  adequate  for  personnel 
assuming  the  CMI  instructor  job.  This  research  requires  specifying  the 
proper  instructor  job  functions,  developing  the  training  materials  for  Marine 
Corps  personnel  assigned  the  CMI  instructor  billet,  and  evaluating/revising 
these  materials. 

A  project  that  will  provide  the  CMI  instructor  role  definition  end  job 
specification  for  the  MCCES  system  has  already  been  planned  and  is  now 
getting  underway.  The  planned  CMI  instructor  training  research  is  to  be 
performed  by  the  McDonnell  Douglas  Corporation  and  is  jointly  funded  by 
h'AVPERSRANDCEN  and  the  Advanced  Research  Projects  Agency  (ARPA) .  Joint 
funding  is  occurring  since  this  project  meets  the  needs  of  the  Navy,  Air 
Force,  and  proposed  Marine  Corps  computer-based  training  systems.  The 
general  CMI  instructor  role  model  and  the  MCCES  CMI  instructor  job  speci¬ 
fication  are  expected  from  McDonnell  Douglas  during  January  1980,  _if_  the 
present  contract  schedule  is  maintained. 

The  instructor  training  research  that  would  be  needed  during  the 
January-August  1980  timeframe,  and  has  not  been  scheduled,  includes  the 
development  and  test  of  the  actual  instructor  training  package  for  Marine 
Corps  CMI  instructors.  It  is  estimated  that  actual  instructor  training 
would  require  about  1  full  week.  Development  of  the  materials  by  a  con¬ 
tractor,  together  with  test  and  revision  of  the  materials,  is  estimated 
to  cost  approximately  $35,000  during  FY  1980. 

It  should  be  noted  that  the  planned  McDonnell  Douglas  contract  will 
only  provide  the  Marine  Corps  CMI  instructor  job  specification  and  not  the 
developed  training  materials.  The  research  recommended  in  this  report  will 
provide  for  the  training  materials  and  necessary  evaluation,  if  the  addi¬ 
tional  funding  can  be  secured. 

Time  Management 

Although  the  CBE  system  computer  will  have  the  capability  for  predicting 
student  completion  times  and  tracking  study  completion  times,  research  is 
needed  to  make  sure  that  the  predictions  are  accurate  and  that  proper  time 
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management  procedures  are  used  in  the  learning  centers.  These  procedures 
keep  tidluiiifc,  Lime  to  a  minimum  and  are  a  major  determinant  of  the  high 
degree  of  cost  avoidance  associated  with  a  successful  CBE  system.  Procedures 
developed  and  used  on  the  Air  Force  CMI  system  have  been  shown  to  succeed 
in  reducing  required  training  time.  These  procedures  could  be  redesigned 
to  fit  the  requirements  of  the  Marine  Corps  and  their  effectiveness  tested 
on  the  CBE  system.  A  contractor  could  perform  this  research,  which  is  es¬ 
timated  to  cost  approximately  $50,000  over  12  months.  Ideally,  this  work 
would  be  performed  during  FY  1981  and  FY  1982,  with  $25,000  allocated  each 
fiscal  year. 

System  Evaluation 

The  final  research  recommendation  necessary  to  ensure  full  system  capabil¬ 
ity  involves  developing  the  system  evaluation  feature  of  the  CBE  system. 
Evaluation  within  the  CBE  context  should  include:  student  achievement  evalu¬ 
ation,  instructional  materials  evaluation,  instructor  evaluation,  CBE  program 
evaluation,  and  training  evaluation  in  terms  of  an  external  measure  of  job 
performance.  Planned  characteristics  of  the  system  will  provide  the  student, 
materials,  and  instructor  evaluation  capabilities.  The  necessary  additional 
research  will  focus  on  developing  the  system's  ability  to  evaluate  itself 
in  terms  of  system  cost-effectiveness  and  job  performance.  This  work  could 
be  performed  by  a  contractor,  and  is  roughly  estimated  at  a  cost  of  $15,000 
for  FY  1980,  $15,000  for  FY  1981,  $25,000  for  FY  1982.  Although  this  work 
is  not  necessary  for  initial  implementation,  the  research  is  required  if 
the  CBE  system  is  to  attain  its  full  capabilities. 

Other  CBE  Research 


Other  research  activities  that  may  play  a  part  in  the  development  of 
the  CBE  system  are  mentioned  below.  Two  of  these  efforts  involve  contri¬ 
butions  by  reserve  Marine  Corps  officers. 

Student  Proctors.  Dr.  Dick  Evans  (COL,  USMCR)  will  be  developing  and 
testing  a  model  for  the  presentation  of  individualized  instruction  during 
the  summer  of  1979.  NAVPERSRANDCEN  has  provided  the  IBM  5110  computer 
necessary  for  the  conduct  of  this  study.  This  study  will  examine  procedural 
changes  which  can  facilitate  self-paced  study  under  competency-based  instruc¬ 
tion  using  student  proctors  to  aid  the  instructor.  Information  from  this 
effort  will  be  used  to  develop  general  procedures  for  guiding  students  under 
computer  management  in  the  individualized  instruction  setting.  NAVPERSRANDCFN 
may  soon  have  complementary  research  underway  in  the  area  of  student  proctor 
use  in  the  Navy  CMI  system.  The  results  could  easily  apply  to  the  Marine 
Corps'  setting. 

System  Acceptance.  Dr.  Lanny  Rope  (LTCOL,  USMCR)  will  be  developing 
documents  to  describe  the  CBE  system  to  the  school  user  community  and  the 
educational  world  at  large.  This  documentation  should  help  gain  MCCES 
training  staff  acceptance  and  provide  information  to  organizations  external 
to  MCCES. 
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Full  Mobilization  Capability.  As  presently  envisioned,  the  MCCES  CBE 
system  will  be  implemented  with  a  student  load  of  1500-2000.  In  the  event 
of  a  national  emergency  requiring  full  mobilization,  this  system  load  could 
increase  to  approximately  5,000.  Prudence  would  appear  to  dictate  that 
a  research  effort  be  undertaken  soon  to  determine  the  exact  augmentations 
necessary  to  enable  the  CBE  system  to  meet  this  surge  in  an  orderly  and 
effective  manner.  Estimates  of  the  required  increments  in  personnel  and 
logistical  support,  together  with  procedural  guidelines  for  securing  these 
in  a  timely  manner,  can  be  arrived  at  by  simulation  and  should  be  documented 
for  easy  access  by  command  personnel.  It  is  estimated  that  a  complete 
product  of  this  kind  could  be  supplied  as  early  as  1983,  or  1  year  after 
the  system  becomes  operational  to  an  extent  sufficient  to  provide  an  accurate 
basis  for  simulation. 

In  conclusion,  the  instructional  materials  development  effort  is  not 
included  in  this  section  as  research;  rather,  it  is  presented  in  the  imple¬ 
mentation  plan  since  it  is  an  absolute  requirement  for  system  implementation. 
The  decision  concerning  the  strategy  to  be  followed  in  developing  instruc¬ 
tional  materials  is  recognized  to  be  partly  economic,  but  it  must  be  pointed 
out  that  abundant  empirical  evidence  already  exists  to  support  the  contention 
that  false  economies  at  this  critical  stage  can  be  devastating  to  the  eventual 
success  of  the  system.  Contractual  development  of  instructional  materials, 
by  professional  developers,  is  considered  vital  to  initial  system  success. 

If  the  first  two  courses  were  professionally  developed,  they  could  then 
serve  as  models  for  development  of  other  materials  by  MCCES  personnel. 


Long-range  CBE  Requirements 


The  implementation  plan  described  in  this  report  will  assist  MCCES 
through  the  next  4  to  5  years  in  developing  the  CBE  system.  This  recommended 
implementation  includes  the  development  of  six  courses  for  CAI  and  the 
remaining  40+-  courses  for  computer  management,  while  they  remain  lecture 
courses.  Long-range  CBE  considerations  should  include  the  instructional 
development  of  those  40+-  courses  for  possible  implementation  onto  the  CAI 
system.  It  would  not  be  surprising  during  the  next  years  if  courses  taught 
at  MCCES  were  changed  and  these  changes  might  alter  the  course  implementation 
schedule.  Certainly,  the  CBE  office  and  MCCES  Command  should  be  flexible 
when  deciding  precisely  which  courses  to  install.  The  needs  of  the  Fleet 
Marine  Force  and  the  training  community,  as  well  as  cost  effectiveness, 
should  be  the  driving  forces  in  selecting  courses  for  implementation. 

As  the  current  CBE  system  matures,  two  pressures  may  be  exerted  on  MCCES 
Command.  The  first  of  these  pressures  involves  the  need  to  provide  documen¬ 
tation  of  the  CBE  system  to  other  Marine  Corps  training  commands  that  would 
want  to  acquire  a  similar  system.  Accordingly,  MCCES  should  develop  a 
documents  file  so  that  other  commands  can  benefit  directly  from  the  MCCES 
experience.  Clearly,  Headquarters  Marine  Corps  should  be  kept  informed 
of  CBE  developments  at  MCCES  so  that  they  can  make  informed  decisions  about 
whether  the  CBE  system  should  be  acquired  and  installed  at  other  Marine 
training  sites.  To  provide  Marine  Corps  Headquarters  with  the  best  infor¬ 
mation  requires  full  development  of  the  evaluation  capability  of  the  system, 
so  that  higher  management  levels  can  see  the  extent  of  cost  effectiveness 
associated  with  CBE. 


The  second  pressure  on  MCCl'S  may  come  from  within  and  would  take  the 
form  of  numerous  requests  tor  adding,  more  courses  onto  the  present  system. 
Although  the  course  implementation  schedule  could  be  adjusted,  it  nay  not 
be  possible  to  add  many  more  courses  onto  the  system  with  its  current  com¬ 
puter  configuration.  Therefore,  future  consideration  may  have  to  be  given 
to  altering  the  computer  configuration  in  order  to  adjust  to  future  training 
requirements.  Additionally,  the  costs  and  capability  of  computer  technology 
are  changing  so  rapidly  that  in  onlv  2  or  3  years  new  devices  of  much  lower 
costs  may  be  available  that  would  significantly  imp  rove  computer  operation. 
It  is  not  possible  to  j  j edict  either  the  practical  options  that  will  be 
available  for  computers  or  the  exact  spec  if icut i ons  for  additional  computer- 
based  training  requirements  within  MCCES  or  in  other  commands. 

The  extent  of  additional  requirements  for  computer-based  training  will 
depend  upon  the  success  of  the  system  recommended  by  this  report.  The 
MCCES  CBE  system  has  the  potential  for  developing  into  the  world's  finest 
large-scale  operational  training  system  that  is  managed  via  computer. 
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APPENDIX  A 


IMPLEMENTATION  PLAN  TASK  STATEMENTS 

Notes : 

1.  The  numerical  priority  rankings  appearing  in  the  task  statements 
are  defined  as  follows: 

•  1  =  Completion  of  task  required  for  i ni tial  system  implementation. 

•  2  =  Completion  of  task  required  for  full  system  capability. 

•  3  =  Completion  of  task  required  for  full  system  implementation. 

2.  The  alphabetical  codes  used  to  indicate  required  level  of  expertise 

on  the  task  statements  are  defined  as  follows: 

•  A  =  Can  be  accomplished  by  MCCES  personnel  presently  available 

within  currently  authorized  assets. 

•  B  =  Can  he  accomplished  by  MCCES  personnel  but  would  require 

augmentation  of  the  MCCES  T/0. 

•  C  =  Can  be  accomplished  by  MCCES  personnel  with  the  assistance 

of  outside  professional  consultants. 

t  D  =  Can  best  be  accomplished  by  a  professional  contractor. 


FUNCTIONAL  OBJECTIVE:  1.0  Enlist  the  Participation  of  Training  Personnel 


ENABLING  TASK  NO.:  1.0  PRIORITY:  1 


TASK  STATEMENT:  Obtain  participation  of  training  personnel  to  ensure  system 

acceptance  through  proper  development  of  system  requirements , 
instructional  materials,  and  system  functioning. 


BEGIN:  Already  Underway  COMPLETE:  Through  life  of  system. 

PERFORMING  ORGANIZATION  ELEMENT:  Personnel  from' management  and  course 

levels  of  training  establishment. 

REQUIRED  EXPERTISE:  Level  A:  Whatever  they  presently  possess. 


Failure  to  enlist  the  support  of  the  training  personnel  from  the 
beginning  will  ensure  failure  of  the  CBE  system.  In  order  to  ensure  acceptance 
of  the  CBE  system,  it  is  vital  to  make  sure  that  the  training  personnel  who 
will  be  using  the  system  have  opportunities  to  provide  information  about  their 
training  requirements,  their  perceptions  of  the  systems  capabilities,  and  the 
adequacy  of  the  materials  and  the  system  operation.  This  participation  must  be 
a  continuing  activity,  not  only  through  system  development,  but  also  through¬ 
out  system  operation.  It  is  only  through  training  personnel  learning  about  the 
system's  capabilities  that  CBE  will  be  recognized  for  the  important  learning 
vehicle  that  it  is. 


ENABLING  TASK  NO.:  1.1 


PRIORITY:  1 


TASK  STATEMENT:  Provide  feedback  to  CBE  office  regarding  perceptions  of 
the  adequacy  of  the  proposed  CBE  system. 

BEGIN:  August  1979  COMPLETE:  August  1979 

PERFORMING  ORGANIZATION  ELEMENT:  Training  personnel  from  the  command  and 

course  level. 

REQUIRED  EXPERTISE:  Level  A 


This  feedback  must  be  solicited  by  the  CBE  office  and  should  seek  not  only 
evaluative  remarks  but  also  suggestions  for  changes  and  criticisms.  In  pre¬ 
sentation  of  the  proposed  CBE  system,  CBE  staff  should  actively  encourage 
participation  by  course  and  command  level  training  personnel  in  whatever 
capacity  is  possible.  At  the  very  least,  written  feedback  should  be  provided 
to  make  sure  that  comments  are  retained  by  the  CBE  staff. 


A- 2 


PRIORITY:  1 


ENABLING  TASK  NO.:  1.2  (sequenced  for  each 

course  to  be  developed  as 

1.2.1 - 1.2.6) 

TASK  STATEMENT:  Training  course  personnel  provide  input  into  the  course  analysis 
portion  of  instructional  development  to  ensure  that  any  needed 
changes  in  training  objectives  will  be  incorporated  into  the  new 
course. 


BEdN:  CCMC-August  1979  and  when-  COMPLETE: 
ever  each  course  begins 
instructional  development. 

PERFORMING  ORGANIZATION  ELEMENT:  Course  training  personnel. 
REQUIRED  EXPERTISE:  Level  A 


Instructional  developers  should  have  a  data  form  available  for  course 
personnel  to  complete.  This  form  should  direct  course  personnel  to  review 
their  training  objectives  and  to  identify  any  objectives  requiring  change. 
Instructional  developers  should  then  review  these  identified  objectives 
with  the  staff  members  to  obtain  a  detailed  description  of  the  required  ob¬ 
jective  and  appropriate  measurement  standards.  There  may  be  objectives  that 
are  obsolete  and  would  drop  out  of  the  POI.  These  should  be  identified  and 
deleted. 


1 


I 

I 


I 

i 


ENABLING  TASK  •NO.:  1 .3  (Sequenced  through  each)  PRIORITY: 

course  to  be  developed  as  task 
1 ,3.1 . 1.3.6) 

TASK  STATEMENT:  Provide  information  and  students  to  assist 
of  individualized  instructional  materials. 


1 


in  validation 


BEGIN:  During  materials  COMPLETE: 

development  phase. 

PERFORMING  ORGANIZATION  ELEMENT:  Training  course  personnel,  and  6-10 

students  from  current  course  or  awaiting 
instruction. 

REQUIRED  EXPERTISE:  Level  A:  Students  should  not  have  experienced 

training  in  the  area  to  be  validated. 


Course  managers  should  provide  information  as  to  perceptions  about 
the  adequacy  of  the  new  instructional  materials.  Developers  should  make 
sure  that  comments  are  recorded  and  appropriate  changes  included  into 
subsequent  materials.  The  students  should  be  used  to  validate  the  materials 
by  proceeding  through  small  sections  of  instruction,  from  approximately  one 
hour  to  a  day  or  two.  Care  should  be  taken  to  obtain  a  measure  of  the  student's 
completion  time  for  each  module  to  assist  in  development  of  student  management 
functions  by  the  CBE  office  staff. 
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ENABLING  TASK  NO.:  1.4 


PRIORITY:  1 


TASK  STATEMENT:  Training  personnel,  from  courses  to  be  implemented  into  the 
CMI  system  only,  should  provide  course  and  testing  infor¬ 
mation  necessary  for  putting  testing  capabilities  into  the 
computer. 


BEGIN:  After  October  1980  COMPLETE: 


‘PERFORMING  ORGANIZATION  ELEMENT:  Course  personnel 


REQUIRED  EXPERTISE:  Level  A 


Course  personnel  should  provide  copies  of  POI  and  estimated  class 
schedules,  training  objectives,  and  tests  that  are  to  be  automatically 
scored  to  the  CMI  instructional  developers.  Assistance  will  probably  be 
needed  by  the  developers  to  alter  existing  test- items  to  make  them  suitable 
for  CMI  implementation.  Developers  should  make  sure  that  they  teach 
the  course  personnel  what  the  CMI  system  will  do  for  training  personnel. 
This  task  should  be  as  instructive  for  the  course  personnel  as  it  is  for 
the  developers. 
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ENABLING  TASK  NO.:  1.5 


PRIORITY:  1 


TASK  STATEMENT:  CCMC  training  personnel  should  provide  reactions  to  the  CBE 
system  implementation  of  their  course  after  its  initial 
tryout  on  the  CAI/CMI  system. 


BEGIN:  August  1980  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  CCMC  training  personnel 

REQUIRED  EXPERTISE:  Level  A 


This  task  should  be  an  opportunity  for  the  CCMC  personnel  to  register 
criticism  about  the  system.  Every  effort  should  be  taken  to  get  the 
course  staff  involved  in  making  the  CBE  system  work.  True  system  success 
will  occur  only  after  the  course  personnel  accept  the  system  as  theirs. 
Particular  attention  should  be  paid  to  altering  management  practices  that  are 
burdensome  on  the  training  staff. 
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ENABLING  TASK  NO.:  1.6 


PRIORITY:  3 


TASK  STATEMENT:  Training  personnel  for  each  course  on  the  system  should  provide 
input  to  the  system  regarding  the  system's  effectiveness  in 
carrying  out  its  training  responsibility. 


BEGIN:  Semi-annually  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  Course  Personnel 


REQUIRED  EXPERTISE:  Level  A 


Personnel  from  courses  on  either  the  CAI  or  the  CMI  systems  should 
provide  information  to  the  CBE  system  about  its  apparent  effectiveness.  The 
computer  system  itself  will  provide  summary  information  to  the  CBE  office  and 
ultimately  to  MCCES  management,  but  course  personnel  should  be  able  to  for¬ 
mally  indicate  to  the  CBE  office  their  level  of  satisfaction  with  the  system 
and  their  suggestions  for  its  improvement. 


FUNCTIONAL  OBJECTIVE:  Design  the  CBE  System 
ENABLING  TASK  NO. :  2.0 

This  objective  is  central  to  the  successful  completion  of  the  entire 
project.  We  have  chosen  to  present  the  system  design  plan  in  two  separate 
sections,  one  dealing  with  design  of  the  instructi onal  system  and  one  deal¬ 
ing  with  design  of  the  computer  system  which  will  support  it. 

Instructional  System.  The  design  of  any  instructional  system  should 
proceed  in  the  light  of  careful  analyses  of  who  is  to  be  taught  to  do 
what  and  how  well.  From  such  analyses,  there  emerges  an  instructional 
technology  that  may  best  be  described  as  a  system  of  interacting  components. 
This  is  the  case  with  the  instructional  mission  of  MCCES,  and  the  first 
section  of  this  system  design  plan  is  devoted  to  an  implementation  plan  for 
specifying  in  considerable  detail  the  design  characteristics  of  the  optimal 
system  for  meeting  the  instructional  requirements  of  ttie  MCCES  CBE  system. 

Computer  System.  In  light  of  the  requirements  for  the  instructional 
system  set  forth  in  the  design  of  that  system,  a  computer  system  as  a 
major  support  element  can  be  designed.  A  plan  for  implementing  this  design 
is  presented  in  the  second  section  of  this  system  design  plan. 

It  is  important  that  these  two  systems  be  kept  conceptually  distinct,  at 
least  initially.  The  computer  should  always  be  viewed  as  a  device  to  aid  and 
support  the  instructional  process,  not  as  a  constraint  which  dictates  solu¬ 
tions  to  problems  of  instructional  design  or  procedure.  By  keeping  the  com¬ 
puter  system  properly  subservient  at  the  design  stage,  there  is  a  greater 
liklihood  that  the  eventually  combined  system  will  serve  the  need  for  which 
it  is  intended:  to  provide  maximally  effective  training  at  the  lowest 
possible  cost  for  USMC  personnel. 

Two  major  recommendations  emerge  from  consideration  of  the  system  design 
plan: 

(1)  A  highly  qualified  systems  analyst  must  be  retained  as  soon  as 
possible  and  be  made  responsible  for  design  and  implementation 
activities  throughout  the  developmental  life  of  the  project. 

(2)  The  entire  computer  system  must  be  planned  and  designed  as  a  unit, 
not  as  a  series  of  sequential  systems  to  be  appended  as  they  are 
developed. 

The  rationale  for  each  of  these  reconmendations  is  discussed  in  detail 
on  the  relevant  task  sheet. 
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FUNCTIONAL  OBJECTIVE:  Design  Instructional  System 


ENABLING  TASK  NO.:  2.1  PRIORITY:  1 

TASK  STATEMENT:  Design  the  instructional  system. 


BEGIN:  July  1979  COMPLETE:  November  1979 


PERFORMING  ORGANIZATION  ELEMENT:  CBE  staff 


REQUIRED  EXPERTISE:  Level  D:  A  highly  qualified,  experienced  consulting 

systems  analyst  (CSA)  with  prior  involvement  in  large 
scale,  computer-based  instructional  technology. 


The  CBE  staff  of  MCCES  has  made  an  excellent  beginning  on  this  overall  task 
by  putting  in  place  a  set  of  procedures  for  determining  user  requirements  of  the 
proposed  system.  It  is  well  known  that  teachers  are  often  resistant  to  change 
and  innovation,  but  the  effects  of  this  resistance  can  be  greatly  lessened  if 
a  sense  of  participation  and  responsibility  is  inculcated  from  the  outset.  We 
believe  an  excellent  beginning  has  been  made  for  insuring  this  outcome  and  that 
the  resulting  system  will  benefit  greatly. 

Before  the  project  proceeds  much  further,  we  regard  as  absolutely  crucial  the 
appointment  of  an  experienced  consulting  system  analyst  (CSA)  as  described  above. 
Designing  a  complex,  computer-based  instructional  system  to  serve  a  multiplicity 
of  courses  and  a  variety  of  support  and  administrative  functions  is  a  task  that 
will  probably  never  be  accomplished  to  perfection.  There  are  always  problems 
with  particular  subsystems  getting  or  storing  the  data  needed  to  perform  its 
function  without  disrupting  the  operation  of  other  subsystems  and  there  is  a 
continuing  need  to  improve  the  interface  of  the  entire  system  to  meet  the  needs 
of  the  human  clientele  it  serves.  Given  that  it  is  impossible  to  design  the 
perfect  system,  the  next  best  thing  is  to  avoid  the  crude  and  costly  errors  teat 
have  plagued  earlier  efforts.  The  surest  way  to  accomplish  this  is  to  arrange 
for  extensive  and  responsible  involvement  by  an  indiviaual  who  is  tnoruuyn'iy 
familiar  with  the  large,  extant  systems,  preferably  someone  who  was  involved 
in  the  design  and  management  of  one  or  more  of  those  systems. 


2]  (Continued) 

We  give  the  highest  priority  on  securing  the  services  of  such  an  individual. 

Too  much  thought,  effort,  and  dedication  has  already  been  expended  on  behalf  of  this 
venture  to  allow  it  to  flounder  because  of  a  lack  of  experienced  design  support. 

Yet  flounder  it  will  if  the  critical  stages  of  design  and  implementation  are  not 
discharged  by  someone  with  the  highest  level  of  skill  and  experience. 


ENABLING  TASK  NO.:  2.1.1  PRIORITY:  1 

TASK  STATEMENT:  Identify  and  contact  services  of  outside  Consulting  Systems 
Analyst  (CSA) 

BEGIN:  Immediately  COMPLETE:  ASAP 

PERFORMING  ORGANIZATION  ELEMENT:  MCCES  Command  and  CBE  Staff 

REQUIRED  EXPERTISE:  Level  A:  Ability  to  evaluate  credentials  and  experience 

of  applicants.  Level  C. 


ESTIMATED  MAN  YEARS:  1  Man-Year  ESTIMATED  COST: 

for  contract  serv' 
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ENABLING  TASK  NO. :  2.1.2 


PRIORITY:  1 


TASK  STATEMENT:  Develop  prel iminary  specification  of  instructional  procedures 
and  design  of  supportabi 1 ity  system. 


BEGIN:  August  1979  COMPLETE:  August  1979 


PERFORMING  ORGANIZATION  ELEMENT:  CBE  staff  and  CSA 


REQUIRED  EXPERTISE:  Level  C:  Knowledge  and  experience  with  complex,  indi¬ 
vidualized  instructional  systems. 


It  is  recognized  that  a  good  portion  of  this  task  has  been  completed  or  is  in 
progress.  It  therefore  serves  as  an  excellent  entry  point  for  the  CSA  to  begin 
acquiring  thorough  familiarization  with  the  existing  instructional  system  ana  with 
the  expectations  of  the  users  of  the  forthcoming  system.  At  this  stage,  all 
proposed  course  delivery  formats,  testing  procedures,  record  keeping  functions, 
and  necessary  managerial  operations  should  be  identified  and  preliminary  system 
descriptions  completed. 


ENABLING  TASK  NO.:  2.1.3 


PRIORITY:  2 


TASK  STATEMENT:  Present  preliminary  instructional  system  design  to  MCCES 
command  personnel . 

BEGIN:  August  1979  COMPLETE:  Same  day 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  Staff  and  CSA 

REQUIRED  EXPERTISE:  Level  c 


This  step  will  insure  that  the  proposed  system  meets  the  expectations  of 
cormand  personnel.  Command  personnel,  in  turn,  can  provide  suggestions  and 
guidelines  in  a  timely  manner.and  thereby  avert  costly  design  errors  arising 
from  misinformation  or  erroneous  assumptions. 
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ENABLING  TASK  NO.:  2.1.4  PRIORITY:  1 

TASK  STATEMENT:  Present  proposed  instructional  system  to  potential  users: 
instructional  staff,  training  managers,  etc. 


BEGIN:  September  1979  COMPLETE:  September  1979 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  staff  and  CSA 

REQUIRED  EXPERTISE:  Level  C 


This  step  is  one  of  the  more  crucial  to  the  long  range  success  of  the  system, 
since  it  must  establish  among  the  operational  community  the  attitude  that  the 
system  will  belong  to  them, and  that  their  input  can  be  instrumental  not  only 
in  its  original  design,  but  in  later  modifications. 

An  immediate  product  of  this  step  will  be  the  elicited  feedback  from  the 
potential  users, which  will  enter  into  the  final  design  of  the  instructional 
system. 

It  is  therefore  critical  that  enough  time  be  taken  to  complete  this  step 
thoroughly  so  that  design  can  proceed  on  the  basis  of  the  best  information 
currently  available. 
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ENABLING  TASK  NO.:  2.1.5 


PRIORITY:  i 


TASK  STATEMENT:  Incorporate  user  feedback  in  revision  of  system  desi 

BEGIN:  September  1979  COMPLETE:  September  1979 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  Staff  and  CSA 

REQUIRED  EXPERTISE:  Level  C 


See  2.1.4.  Designers  must  evaluate  user  feedback  and  redesign 
the  system  in  accordance  with  relevant  comments  and  suggestions. 


FUNCTIONAL  OBJECTIVE: 


ENABLING  TASK-NO.:  2.1.6  PRIORITY:  1 

TASK  STATEMENT:  Formalize  instructional  system  by  preparing  a  document 

describing  the  precise  instructional  features  of  the  system. 

BEGIN:  September  1979  COMPLETE:  September  1979 

PERFORMING  ORGANIZATION  ELEMENT:  CSA  and  CBE  staff 
REQUIRED  EXPERTISE:  Level  C 


This  step  will  yield  a  product  that  is  a  formal  description  of  the 
entire  instructional  system  as  envisaged.  It  will  include  not  only  the 
detailed  presentation  of  the  methods  and  procedures  to  be  used  in  the  six 
courses  intended  for  conversion  to  CAI/CMI  delivery,  but  will  specify  those 
procedures, such  as  automated  test  scoring  and  student  management,  that  will 
be  added  to  courses  designed  for  CMI  implementation  only. 
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ENABLING  TASK  NO.: 


2.1.7 


PRIORITY:  1 


TASK  STATEMENT:  Define  and  document  the  computer  capabilities  required  to 
support  the  instructional  system. 


BEGIN:  October  1979 
PERFORMING  ORGANIZATION  ELEMENT: 
REQUIRED  EXPERTISE:  Level  C 


COMPLETE:  October  1979 
CSA  and  CBE  staff 


i 


This  step  is  necessary  to  make  explicit  the  computer  system  requirements 
and  capabilities  necessary  to  bring  the  instructional  system  formalized 
in  2.1.6  into  existence.  Only  by  formalizing  this  step  can  we  be  assured 
that  the  proposed  system  has  the  capability  of  doing  exactly  wnat  is  expected 
by  the  users  and  designers. 


ENABLING  TASK  NO.:  2.1.8 


PRIORITY:  1 


TASK  STATEMENT:  Finalize  instructional  system  design  by  documenting 
instructional  attributes  of  CBE  system. 


BEGIN:  November  1979  COMPLETE:  December  1979 


PERFORMING  ORGANIZATION  ELEMENT:  CSA  and  CBE  staff 


REQUIRED  EXPERTISE:  Level  C 


This  step  integrates  the  products  of  2.1.6  and  2.1.7  and  yields  a  final 
document  that  will  serve  as  the  basic  reference  throughout  the  development 
phase.  It  is  essential  that  this  document  be  prepared  with  both  clarity  and 
completeness  so  that  it  may  serve  as  a  model  for  other  CBE  systems  contemplated 
elsewhere  in  the  Marine  Corps  training  establishment. 


ENABLING  TASK  NO.: 


2.2 


PRIORITY:  1 


TASK  STATEMENT:  Determine,  design,  and  document  the  needs  of  each  of 
several  computer  subsystems. 


BEGIN:  December  1979  COMPLETE:  March  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CEB  Staff  and  CSA 

REQUIRED  EXPERTISE:  Levels  C  and  D 


We  have  already  commented  on  the  general  need  for  a  highly  competent,  ex¬ 
perienced  systems  analyst  to  coordinate  the  design  and  development  of  the 
entire  system.  That  need  is  manifest  in  the  major  modification  of  the  design 
process.  The  CBE  staff  did  an  excel  lent  job  of  identifying  the  major  subsystems 
required,  and  preparing  a  well  sequenced  implementation  plan  for  insuring 
their  development.  Unfortunately,  experience  has  shown  that  all  of  the  sub¬ 
systems  must  be  designed  very  nearly  at  the  same  time  or  insurmountable  inter¬ 
face  problems  will  eventually  surface.  The  Navy  CMI  system  is  a  good  example 
of  this;  failure  to  anticipate  and  provide  the  data-base  management  needs  of 
the  extensive  response  history  files  has  left  these  data  all  tut  inaccessible 
for  the  variety  of  management  and  research  functions  that  it  is  now  recognized 
they  should  be  serving.  This  is  in  no  way  a  criticism  of  the  Navy  system;  it 
was  not  designed  from  the  outset  to  serve  these  functions,  but  modifying  it  now 
to  meet  these  additional  needs  is  a  costly  process  that  should  be  avoided  in  the 
case  of  the  MCCES  system. 

Problems  of  this  type  are  easily  avoided  by  designing  the  major  system  and 
its  participating  subsystems  as  an  integral  whole,  permitting  the  solution  of 
interface  problems  to  be  accomplished  before  huge  data  bases  have  teen  accu¬ 
mulated.  Accordingly,  one  of  our  proposed  major  moeifications  to  the  original 


2.2  (continued) 


implementation  plan  becomes  apparent  when  one  locks  at  the  time  frames  for 
designing  the  several  subsystems  described  in  the  following  pages:  they 
are  all  identicall  Moreover,  the  three-month  span  i ndi cated  wi 11 ,  we 
believe,  enable  the  system  to  be  functioning  by  the  target  date  of  August 
1980.  This  may  place  severe  strain  on  the  resources  of  the  design  personnel. 

If  so,  the  recommendation  regarding  sacrifice  of  design  criteria  for  the  sake 
of  advancing  the  date  of  initial  operation  should  be  obvious:  don  *t--the 
eventual  price  is  unjustified. 

The  several  subsystems  described  in  the  following  pages  are  meant  only 
as  a  suggested  constellation  that  will  meet  all  of  the  required  functions  of 
the  envisioned  system.  This  list  is  not  meant  to  constrain  the  designers 
in  any  way.  The  eventual  list  of  subsystems  may  be  longer  or  shorter  de¬ 
pending  on  how  various  functions  are  combined;  and  this,  in  turn,  will  depend 
to  some  degree  on  the  architecture  and  operating  characteristics  of  the  system 
that  is  eventually  procured.  We  offer  this  list  merely  to  document  the  fact 
that  provision  must  be  made  for  the  performance  of  all  the  functions  described; 
how  that  is  to  be  best  accomplished  will  be  a  decision  properly  made  by  the 
consulting  systems  analyst. 


ENABLING  TASK  NO.:  2.2.1 


PRIORITY:  ] 


TASK  STATEMENT:  Merge  final  instructional  system  requirements  with  system 
capabilities  enumerated  on  solicitation  document. 

BEGIN:  December  1979  COMPLETE:  December  1979 

PERFORMING  ORGANIZATION  ELEMENT:  CSA  and  CBE  staff 

REQUIRED  EXPERTISE:  Level  C 


This  is  the  initial  step  in  the  design  of  the  computer  system.  Any  in¬ 
abilities  of  the  planned  system  to  meet  the  requirements  of  the 
instructional  system  must  be  resolved  by  either  (a)  amending  the  requirement  from 
the  instructional  side,  or  (b)  amending  the  contract  with  the  system  vendor. 

The  result  of  this  effort  will  be  a  specification  of  the  capabilities  of  the 
solicited  system  and  how  each  will  serve  the  various  functions  identified  in 
the  final  instructional  system  statement. 
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ENABLING  TASK  NO.: 


2.2.2 


PRIORITY:  1 


TASK  STATEMENT:  Determine  and  document  requirements  for  the  data  base 
management  subsystem. 

BEGIN:  January  1980  COMPLETE:  January  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CS/VCBE  staff' 

REQUIRED  EXPERTISE:  Levels  C  and  D 


Any  computer-based  instructional  system  has  the  potential  for  generating 
a  very  large  amount  of  data,  nearly  all  of  it  useful  in  either  managing  or 
improving  the  system.  Unfortunately,  most  designers  discover  this  fact  after 
it  is  too  late  to  make  use  of  it,  and  vast  quantities  of  data  are  either  dis¬ 
carded  or  rendered  all  but  inaccessible  by  improper  data  base  design  and 
management  practices. 

In  order  that  the  MCCES  CBE  system  might  be  among  the  first  major  systems 
to  avert  this  difficulty,  we  are  recommending  that  special  care  be  exercised 
at  this  step  and  that  it  be  accomplished  with  great  thoroughness.  We  suggest 
that  the  staff  avail  themselves  of  the  experience  of  other  designers  and 
managers  of  major  instructional  systems,  and  heed  well  their  cautions  concerning 
those  data  base  elements  that  might  otherwide  be  overlooked  or  neglected.  The 
long  term  utility  of  the  system  will  profit  immeasurably  from  extra  care  taken 
at  this  time. 
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ENABLING  TASK  NO.:  2.2.3  PRIORITY:  1 

TASK  STATEMENT :  Design  the  data  base  management  subsystem. 

BEGIN:  January  1980  COMPLETE:  March  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CSA/CBE  staff 

REQUIRED  EXPERTISE:  Level  C 


This  task  includes  those  activities  normally  associated  with  the 
design  of  a  complex  computer  subsystem. 
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ENABLING  TASK  NO.: 


2.2.4 


PRIORITY:  1 


TASK  STATEMENT:  Determine  and  document  requirements  for  tne  course  management 
subsystem. 

BEGIN:  January  1980  COMPLETE:  March  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CSA/CBE  staff 

REQUIRED  EXPERTISE:  Level  C 


i 

The  course  management  subsystem  will  perform  such  functions  as:  generate  ' 
class  rosters,  classify  and  register  students  on  the  basis  of  information  j 
available  from  the  MMS  tapes,  generate  grade  reports,  and  generate  course  : 
evaluation  data.  In  addition  to  instructors,  users  such  as  staff  data  analysts  j 
should  be  consulted  to  determine  the  nature  and  frequency  of  their  need  for  j 
any  of  the  foregoing  classes  of  information. 


I 


ENABLING  TASK  NO.:  2.2. 5  PRIORITY:  1 

TASK  STATEMENT:  Design  the  course  management  subsystem. 

BEGIN:  January  1980  COMPLETE:  March  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CSA/CBE  Staff 

REQUIRED  EXPERTISE:  Level  C 


ENABLING  TASK  NO.: 


2.2.6 


PRIORITY:  1 


TASK  STATEMENT:  Determine  and  document  requirements  for  the  student  management 
subsystem 


BEGIN:  January  1980  COMPLETE:  March  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CSA/CBE  staff 

REQUIRED  EXPERTISE:  Level  C 


This  subsystem  wi 1 1  perform  the  following  functions:  (a)  determine 
predicted  completion  times  for  each  student  on  each  module  or  unit  of 
training  material,  (b)  track  individual  student  progress  through  a  given 
course  or  cojrses,  (c)  schedule  and  assign  students  to  courses  or 
other  instructional  activities,  and  (d)  generate  individual  student 
evaluation  reports. 

It  will  undoubtedly  be  useful  at  this  stage  to  draw  on  the  experience 
of  individuals  actively  involved  with  managing  these  functions  in  the 
Navy  and/or  Air  Force  CBE  systems  to  insure  that  no  major  functions 
are  neglected. 
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ENABLING  TASK  NO.: 


2.2.1 


PRIORITY- 


TASK  STATEMENT:  Design  the  student  management  subsystem. 

BEGIN:  January  1980  COMPLETE:  March  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CSA/CBE  staff 


REQUIRED  EXPERTISE:  Level  C 


ENABLING  TASK  NO.: 


2.2.8 


PRIORITY:  I 


TASK  STATEMENT:  Determine  and  document  requirements  for  the  staff  management 
subsystem. 

BEGIN:  January  1980  COMPLETE:  March  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  staff/CSA 

REQUIRED  EXPERTISE:  Level  C 


Functions  to  be  performed  by  this  subsystem  include  generation  of  staff 
rosters,  scheduling  of  instructors  and  support  personnel  to  courses,  and 
instructor  evaluation.  Complete  determination  of  the  requirements  for  this 
subsystem  may  have  to  await  the  completion  of  research  on  the  role  and  function 
of  the  CMI/CAI  instructor  so  that  meaningful  evaluation  procedures  can  be 
designed,  but  provision  for  this  function,  as  for  all  other  functions,  must 
be  made  in  the  initial  design  stages. 
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ENABLING  TASK  NO.: 


2.2.9 


PRIORITY:  1 


TASK  STATEMENT: 


Design  the  staff  management  subsystem. 


BEGIN:  January  1980  COMPLETE:  March  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CSA/CBE  staff 
REQUIRED  EXPERTISE:  Level  B 
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ENABLING  TASK  NO.: 


2.2.10 


PRIORITY:  1 


TASK  STATEMENT:  Determine  and  document  requirements  of  the  resource  allocation 
subsystem. 


BEGIN:  January  1980  COMPLETE:  March  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  Staff/CSA 


REQUIRED  EXPERTISE:  Level  C 


ENABLING  TASK  NO. :  2.2.11 


PRIORITY 


TASK  STATEMENT:  Design  the  resource  allocation  subsystem. 

BEGIN:  January  1980  COMPLETE:  March  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CSA/CBE  Staff' 

REQUIRED  EXPERTISE:  Level  C 
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ENABLING  TASK  NO.:  2.2.12 


PRIORITY:  1 


TASK  STATEMENT:  Determine  and  document  requirements  of  the  response  history 
subsystem. 

BEGIN:  January  1980  COMPLETE:  March  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  Staff/CSA 

REQUIRED  EXPERTISE:  Level  C 


This  subsystem  is  at  the  heart  of  many  management,  and  virtually  all  evalu¬ 
ative  functions,  that  will  be  performed  by  the  system.  It  provides  for  the 
storage  and  retrieval  of  the  details  of  each  student's  interaction  with  his 
course(s)  and  with  other  facets  of  the  instructional  system  (judicial  boards, 
etc.).  Because  not  all  courses  will  be  fully  CAI/CMI,  special  care  must  be 
taken  to  make  the  records  of  those  students  taking  courses  having  only  minimal 
computer  assistance  compatible  with  those  of  students  in  the  fully  implemented 
CBE  courses.  A  major  category  of  users  of  the  response  history  subsystem  will 
be  members  of  the  research/analysis  community;  it  is  anticipated  that  DoD 
personnel  will  avail  themselves  of  the  opportunity  to  query  the  contents  of 
this  subsystem  if  indeed,  as  we  project,  the  MCCES  CBE  system  becomes  the 
prototype  of  its  kind  in  the  military  community. 


ENABLING  TASK  NO.:  2.2.13 


PRIORITY:  1 


TASK  STATEMENT:  Design  the  response  history  suosysten . 

BEGIN:  January  1980  COMPLETE:  March  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CSA  /CBE  Staff 
REQUIRED  EXPERTISE:  Level  C 


Because  this  subsystem  is  central  to  so  many  other  functions  of  the  system 
(see  2.2.12),  it  is  essential  that  its  design  facilitate  rapid  access  by  the 
other  subsystems.  At  the  same  time,  this  subsystem  will  generate  the  largest 
volume  of  data  per  unit  time,  placing  special  demands  on  the  data  base  management 
capabilities  that  apply  to  these  data  sets.  These  problems  are  currently  being 
addressed  by  individuals  in  the  Military  Information  and  Instructional  Systems 
Activity  (MIISA)  located  at  NAS  Memphis  and  it  is  recommended  that  their  design 
solutions  be  consulted. 
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ENABLING  TASK  NO.: 


2.2.14 


PRIORITY:  1 


TASK  STATEMENT:  Determine  and  document  requirements  of  the  instructional 
management  subsystem. 

BEGIN:  January  1980  COMPLETE:  March  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  Staff/  CSA 

REQUIRED  EXPERTISE:  Level  C 


The  instructional  management  subsystem  performs  the  basic  instructional 
functions  of  the  CBE  system,  from  the  terminal-mediated  drill  and  practice 
functions  characteristic  of  the  full  CAI  mode,  to  the  lesson  assignment, 
testing,  evaluating,  remediating,  and  reassigning  functions  typical  of  a 
CMI  system.  This  subsystem  interacts  continuously  with  the  response  history 
subsystem  and  is  also  the  principal  interface  between  the  system  and  the 
student.  For  this  reason,  subsystem  reliability  is  a  critical  design  con¬ 
sideration,  to  be  determined  by  analysis  of  student  contact  frequency  and 
duration,  and  potential  training  time  losses  attendant  to  subsystem  failure. 
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ENABLING  TASK  NO.:  2.2.15 


PRIORITY:  1 


TASK  STATEMENT:  Design  the  instructional  management  : 

BEGIN:  January  1980  COMPLETE:  March 

PERFORMING  ORGANIZATION  ELEMENT:  CSA/CBE  Staff 
REQUIRED  EXPERTISE:  Level  -C 
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ENABLING  TASK  NO.: 


2.2.16 


PRIORITY:  1 


TASK  STATEMENT:  Determine  and  document  the  instructional  materials  development, 
editing,  and  revision  subsystem. 


BEGIN:  January  1980  COMPLETE:  March  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  Staff/ CSA 

REQUIRED  EXPERTISE:  Level  C 


The  essentials  of  this  system  are  to  be  provided  by  the  vendor;  it  is 
necessary  to  insure  at  this  step  that  the  needs  of  the  instructional 
system  will  be  served  by  the  computer  system  as  acquired. 


ENABLING  TASK  NO.: 


2.2.17 


PRIORITY:  1 


TASK  STATEMENT:  Design  the  instructional  materials 'development,  editing,  and 
revision  subsystem. 


BEGIN:  January  1980  COMPLETE:  March  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CSA/CBE  Staff 
REQUIRED  EXPERTISE:  Level  C 


Provide  revisions  to  the  vendor's  subsystem  that  are  necessary  to 
accommodate  any  unique  or  idiosyncratic  features  of  the  lessonware 
contemplated  by  the  instructional  system. 


ENABLING  TASK  NO.:  2.2.18  PRIORITY:  1 

TASK  STATEMENT:  Determine  and  document  the  instructional  materials  evaluation 
system. 

BEGIN:  January  1980  COMPLETE:  March  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  Staff/  CSA 

REQUIRED  EXPERTISE:  Level  C 


Again,  this  function  is  generally  served  by  the  vendor's  subsystem,  but 
a  major  elaboration  is  indicated  by  experience  with  both  the  Air  Force  and 
Navy  CMI  systems.  Specifically,  in  large  systems  such  as  those  serving 
training  pipelines  (sequenced  courses)  it  is  both  possible  and  desirable  to 
evaluate  portions  of  the  instructional  materials  in  terms  of  the  performance 
of  students  in  follow-on  courses  for  which  the  subject  materials  are  pre¬ 
requisite.  This  function  relies  heavily  on  the  response  history  subsystem 
and  must  be  planned  accordingly;  the  rapid  identification  of  defective  por¬ 
tion-,  of  the  instructional  materials  and  their  prompt  removal  and/or  revision 
is  a  function  few  instructional  systems  can  support,  but  one  which  augurs 
for  substantial  savings  in  total  throughput  time. 
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ENABLING  TASK  NO.:  2.2.19 


PRIORITY:  1 


TASK  STATEMENT:  Design  instructional  materials  evaluation  system. 

BEGIN:  January  1980  COMPLETE:  March  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CSA'/CBE  Staff 
REQUIRED  EXPERTISE:  Level  C 


ENABLING  TASK  NO. : 


2.2.20 


PRIORITY:  1 


TASK  STATEMENT:  Determine  and  document  requi rements  for  instructional  system 
evaluation  subsystem. 

BEGIN:  Jaruary  1980  COMPLETE:  March  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  Staff/CSA 

REQUIRED  EXPERTISE:  Level  C 


Formal  clannels  for  collecting  evaluations  of  MCCES  graduates'  performance 
in  the  field  should  be  established  and  a  provision  made  for  incorporating  these 
data  in  a  routine  evaluation  of  the  entire  CBI  system.  Additionally,  the 
subsystem  should  routinely  report  course-to-course  performance  conditions  so 
that  adjustments  in  beginning  course  requirements  can  be  made  and  validated 
in  terms  of  later  achievement. 


ENABLING  TASK  NO.: 


2.2.21 


PRIORITY:  1 


TASK  STATEMENT:  Design  instructional  system  eval uation  subsystem. 


BEGIN:  January  1980  COMPLETE:  March  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CSA/CBE  Staff 
REQUIRED  EXPERTISE:  Level  C 


L NAB LING  TASK  NO.:  2.2.22  PRIORITY:  2 

TASK  STATEMENT:  Determine  and  document  requirements  for  on-line  cost- 
effectiveness  monitoring  subsystem 

BEGIN:  January  1980  COMPLETE:  March  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CAS/CBE  Staff 

REQUIRED  EXPERTISE:  Levels  C  or  D 


Command  personnel  responsible  for  training  functions  in  the  military  are 
responsible  for  maximizing  the  output  of  well-trained  personnel,  while  mini¬ 
mizing  or  maintaining  the  costs  of  carrying  out  this  mission.  Unfortunately , 
they  seldom  have  at  their  fingertips  sensitive  data  on  the  momentary  cost- 
effectiveness  of  the  systems  they  command,  and  are  forced  to  rely  on  dated 
information  that  is  often  vague  and  subjective  when  attempting  to  isolate  the 
cause  of  a  sudden  surge  in  costs  or  drop  in  effectiveness. 

The  proposed  MCCES  CBE  system,  with  its  complete  array  of  subsystems,  could 
present  the  first  operational  solution  to  the  above  problem.  By  identifying 
the  collective  sources  of  training  costs  and  monitoring  their  levels,  and  by 
continuously  monitoring  the  effectiveness  of  the  training  system  in  terms  of 
student  achievement  and  pace,  indicators  of  momentary  cost-effectiveness  could 
be  compiled  and  displayed  to  command  personnel  on  demand.  Should  an  undesirable 
situation  or  trend  be  detected,  the  system  could  be  systematical ly  queried  until 
the  probable  source  was  identified.  Implementing  this  feature,  while  not 
essential  to  the  operation  of  the  proposed  system,  would  make  the  system  a  proto¬ 
type  with  regard  to  this  level  of  managerial  control  and  would  provide  a  con¬ 
tinuous  rebuttal  to  those  who  persist  in  the  assertion  that  CBE  is  less  cost- 
effective  than  traditional  a  1 terna t i ves ,  and  who  cite  the  lack  of  any  systematic 
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2.2.22  (continued) 


cost-effectiveness  analysis  in  support  of  their  position.  Because  the  pro¬ 
posed  system  is  a  mix  of  CBE  and  conventional  procedures  augmented  by 
computer  management  capabilities,  a  comparison  is  continuously  available 
and  should  become  part  of  the  information  base  for  informed  decision  and 
pol icy  making. 


I 
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ENABLING  TASK  NO.:  2.2.23 


PRIORITY:  2 


TASK  STATEMENT:  Design  on-line  cost-effectiveness  monitoring  subsystem. 

BEGIN:  January  1980  COMPLETE:  March  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CSA  /CBE  Staff 
REQUIRED  EXPERTISE:  Levels  C  and  D 
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FUNCTIONAL  OBJECTIVE:  Develop  the  Computer  Software  System 


ENABLING  TASK  NO.:  3.0  PRIORITY:  1 

TASK  STATEMENT:  Develop  the  computer  software  system  necessary  to  enable 
the  CBE  system  to  perform  its  intended  instructional 
functions . 


BEGIN:  July  1979  COMPLETE:  July  1980 


PERFORMING  ORGANIZATION  ELEMENT:  CBE  Staff,  CSA  ,  and  support  personnel 


REQUIRED  EXPERTISE:  Levels  A,  B,  C,  D.  Most  of  the  effort  will  be  provided 

by  programmers  on  the  CBE  staff,  with  direct  guidance 
by  the  in-house  systems  analysts  and  the  CAS. 


This  functional  objective  is  designed  to  result  in  the  development  of  the 
computer  software  system  necessary  to  support  the  CBE  instructional  functions. 
Completion  of  this  objective  involves  not  only  performing  and  documenting 
the  applications  programs,  but  also  developing  the  system  capability  of  per¬ 
forming  appropriate  processing  functions.  Although  a  straight-forward 
objective,  the  work  represented  by  this  phase  of  the  plan  is  crucial  to  actual 
operation  of  the  system.  The  proof  of  successful  conduct  of  these  enabling 
tasks  will  be  actual  operation  of  the  CBE  system,  scheduled  for  July  1980. 
Specification  of  the  COBOL  language  in  the  RFP  requires  the  development  of 
application  programs  in  that  language.  Existing  and  anticipated  programming 
resources  should  be  able  to  accomplish  this  programming  activity. 


ENABLING  TASK  NO.:  3.1 


PRIORITY:  1 


TASK  STATEMENT:  Identify  and  train  applications  programmers. 


BEGIN:  July  1979  COMPLETE:  October  1979 

PERFORMING  ORGANIZATION  ELEMENT:  CSA *  CBE  Staff 
REQUIRED  EXPERTISE:  Level  C 


It  is  anticipated  that  these  personnel  will  be  recruited  mainly  from 
an  available  pool  of  military  personnel.  A1 ternati vely ,  if  a  major  contract 
is  let  to  implement  the  system  design  and  development  phases,  the  contractor 
should  assume  responsibility  for  performing  this  step  and  certifying  the 
competence  of  each  hire  to  perform  the  needed  programming  duties. 


ENABLING  TASK;  NO.  :  3.2 


PRIORITY:  i 


TASK  STATEMENT:  Draft  Integrated  Logistics  Support  Plan  (I.L.S.P.) 


BEGIN:  September  1979  COMPLETE:  October  1979 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  Staff  * 

REQUIRED  EXPERTISE:  Level  A 


This  step  can  begin  as  soon  as  the  instructional  system  design  is 
completed. 


I 


ENABLING  TASK,  NO.  :  3.3 

TASK  STATEMENT:  Revise  and  submit  I.L.S.P. 

BEGIN:  October  1979  COMPLETE : 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  Staff 
REQUIRED  EXPERTISE:  Level  A 


PRIORITY:  1 


October  1979 


ENABLING  TASK  NO.:  3.4.1 


PRIORITY:  ] 


TASK  STATEMENT:  Train  application  programmers  to  use  system  acquired  in  3.3.1 

BEGIN:  August  1979  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  Staff 
REQUIRED  EXPERTISE:  Level  A 
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ENABLING  TASK,  NO.  :  3.4 


PRIORITY:  1 


1ASR  STATEMENT:  Acquire  interactive  COBOL  capability. 

BEGIN:  July  1979  COMPLETE:  ASAP 

PERFORMING  ORGAN  I /ATI  UN  ELEMENT:  CBE  Staff 
REQUIRED  EXPERTISE:  Level  A 


Specification  of  COBOL  as  the  computer  system  language  imposes  the  con¬ 
straint  that  all  system  routines  be  written  in  COBOL.  For  purposes  of 
development,  testing,  and  debugging,  a  small,  interactive  mi crosystem  with 
COBOL  capability  will  greatly  accelerate  the  rate  of  program  development. 


ENABLING  TASK  NO. :  3.4.2 


PRIORITY:  1 


TASK  STATEMENT:  Develop  programming  standards. 


BEGIN:  July  1979  COMPLETE:  August  1979 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  Staff/  CSA 
REQUIRED  EXPERTISE:  Level  C 


Detailed 
properly  and 


guidelines  for  programmers  will 
simplify  documentation. 


ensure  that  subroutines  mesh 


i 


ENABLING  TASK  NO.: 


3.4.3 


1 


;  c  r  :  r  .■ 

TASK  STATEMEU1:  Develop  documentation  standards 

BEGIN:  July  1979  COMPLETE :  August  1979 

PERFORMING  ORGANIZATION  ELEMENT :  CBE  Staf f/CSA/Outs i de  Consultant 
REQUIRED  EXPERTISE:  Levels  C  and  D 


This  step  is  critical  if  the  system  is  to  have  any  applicability  beyond 
the  boundaries  of  MCCES  and  the  tour  of  duty  of  the  original  developers. 
Further,  system  test  and  debugging  will  be  greatly  facilitated  if  this  step 
is  carefully  executed. 
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ENABLING  TASK.  NO.  :  3.4.4 


PRIORITY:  1 


TASK,  STATEMENT:  Develop  system  test  plan 

BEGIN:  October  1979  COMPLETE:  Ongoing 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  Staff/CAS 
REQUIRED  EXPERTISE:  Level  C 


Subsystem  tests  will  be  conducted  as  a  routine  part  of  development. 
Interface  tests  must  be  performed  as  pairs  of  subsystems  become  operational 
and  a  master  plan  for  documenting  the  results  of  these  tests  is  desirable 
as  an  adjunct  to  final  system  testing  and  debugging  (See  Task  3.6). 


ENABLING  TASK.  NO.  :  3.b  PRIORITY:  1 

TASK  STATEMENT:  Develop  subsystems. 

BEGIN:  January  1980  COMPLETE:  July  1980 

PERFORMING  ORGANIZATION  ELEMENT:  Staff/CSA 
REQUIRED  EXPERTISE:  Levels  B  and  C 


Development  of  several  subsystems  described  in  Task  2.1  can  and  should 
proceed  in  harmony  with  the  design  process  since  problems  and  constraints 
encountered  in  the  design  activity  will  (must)  be  accommodated  in  the 
development  phase.  We  therefore  perceive  design  and  development  as  essentially 
contemporaneous .  The  order  of  subsystem  development  tasks  contained  in  the 
following  pages,  then,  is  not  an  attempt  to  sequence  the  developmental  effort; 
rather,  it  reflects  our  judgement  as  to  the  priorities  that  should  be  attached 
to  the  various  subsystems.  Extra  care  and  resources,  if  available,  should  be 
directed  to  the  development  of  those  subsystems  appearing  early  in  the  sequence. 
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ENABLING  TASK  NO.:  3.5.1  PRIORITY:  1 

TASK  STATEMENT:  Begin  developing  data  base  management  subsystem. 


BEGIN:  January  1980 


COMPLETE:  July  1980 


ENABLING  TASK  NO.: 


3.5.2 


PRIORITY:  1 


TASK  STATEMENT:  Develop  response  history  subsystem. 

BEGIN:  January  1980  COMPLETE:  July  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  Staff/I  CSA 
REQUIRED  EXPERTISE:  Levels  B  and  C 
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ENABLING  TASK,  NO.: 


3.5.3 


PRIORITY:  1 


TASK  STATEMENT:  Develop  instructional  management  subsystem. 

BEGIN:  January  1980  COMPLETE:  July  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  Staff/'CSA 
REQUIRED  EXPERTISE:  Level  B,  C 
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ENABLING  TASK  NO. : 


3.5.4 


PRIORITY:  1 


TASK  STATEMENT:  Develop  student  management  subsystem. 


BEGIN:  January  1980  COMPLETE:  July  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  Staff/CSA 
REQUIRED  EXPERTISE:  Level  B,  C 


ENABLING  TASK  NO.:  3.5.5 


PRIORITY:  1 


TASK  STATEMENT:  Develop  Course  Management  Subsystem. 

BEGIN:  July  1979  COMPLETE:  July  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  Staff/CSA 
REQUIRED  EXPERTISE:  Level  B,  C 


r  f 


ENABLING  TASK  NO. :  3.5.5. 1  PRIORITY:  1 

TASK  STATEMENT:  Develop  MMS  tape  system. 

BEGIN:  July  1979 
PERFORMING  ORGANIZATION  ELEMENT: 

REQUIRED  EXPERTISE:  Leve1  A*  c 


COMPLETE:  July  1980 
CBE  Staff/ CSA 


This  subobjective  combines  objectives  6.2.1,  6.2.2  and  6.2.3  of  the  original 
MCCES  CBE  implementation  plan  and  concerns  developing  system  compatibility  with 
the  MMS  tape  system.  As  such,  this  function  resides  in  the  registration 
routine  of  the  course  management  subsystem. 

Since  accomplishing  this  objective  will  depend  on  cooperative  interaction 
between  MCCES  and  HQMC,  it  is  suggested  that  efforts  are  begun  now  to  make  the 
arrangements  necessary  to  insure  reliable  access  to  the  MMS  tapes  and 
that  these  arrangements  be  documented  and  secured.  An  embarrassing  source  of 
system  failure  and  time  loss  arises  when  students  are  forced  to  wait  to  register 
for  a  course  because  some  element  external  to  the  system  has  failed  to  per¬ 
form. 
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ENABLING  TASK  NO. :  3.5.6 


PRIORITY:  1 


TASK  STATEMENT:  Develop  materials  development  subsystem. 

BEGIN:  July  1979  COMPLETE:  FEB  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  Staff/Vendor 
REQUIRED  EXPERTISE:  Level  C,  D 


The  timeframe  for  this  subobjective  is  foreshortened  to  insure  that  the 
instructional  materials  development  personnel  have  ample  time  to  author  and 
code  materials  into  the  CBE  system. 
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ENABLING  TASK  NO.:  3. 5. 6.1 


PRIORITY:  1 


TASK  STATEMENT:  Develop  instructional  materials  revision  system. 

BEGIN:  July  1979  COMPLETE:  FEB  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  Staff/Vertdor 
REQUIRED  EXPERTISE:  Level  B  or  C 


ENABLING  TASK  NO.: 


3.5.7 


PRIORITY:  1 


E  TASK  STATEMENT:  Develop  resource  allocation  subsystem. 


BEGIN:  January  1980  COMPLETE:  July  1980 


PERFORMING  ORGANIZATION  ELEMENT:  CBE  Staff/CSA 


ENABLING  TASK  NO.: 


3.5.9 


PRIORITY:  1 


TASK  STATEMENT:  Develop  instructional  materials  evaluation  subsystem. 

BEGIN:  January  1980  COMPLETE:  July  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  Staff/ CSA 
REQUIRED  EXPERTISE:  Level  B,  C 
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ENABLING  TASK  NO.:  3.5.10 


PRIORITY 


TASK  STATEMENT:  Develop  evaluation  subsystem. 

BEGIN:  January  1980  COMPLETE:  July  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  Staff/CSA 
REQUIRED  EXPERTISE:  Level  B,  C 
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ENABLING  TASK  NO.: 


3.5.11 


PRIORITY:  2 


TASK  STATEMENT:  Develop  cost  effectiveness  monitoring  subsystem 


BEGIN:  January  1980  COMPLETE:  January  1981 

PERFORMING  ORGANIZATION  ELEMENT:  CSA'/Consul tant 
REQUIRED  EXPERTISE:  Level  C,  0 


The  time  frame  has  been  expanded  on  this  objective  because  it  cannot  be 
tested  until  the  system  is  operational  and  real  data  are  being  processed. 
Reaction  to  the  information  yielded  by  the  cost-effectiveness  monitoring  sub 
system  can  be  assessed. 
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ENABLING  TASK  NO. :  3.6 

TASK  STATEMENT:  Test  entire  system. 

BEGIN:  July  1980  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  Staff/CSA 


PRIORITY 


REQUIRED  EXPERTISE:  Level  C 


FUNCTIONAL  OBJECTIVE:  Operate  the  CBE  System 

ENABLING  TASK  HO. :  4.0  PRIORITY:  1 

TASK  STATEMENT:  Install  computer  programs  and  courses  onto  the  system  and 
operate  the  CBE  system. 


BEGIN:  As  early  as  October  1979  COMPLETE:  Through  system  operational  life. 
PERFORMING  ORGANIZATION  ELEMENT:  CBE  Staff 


REQUIRED  EXPERTISE:  Levels  B,  C:  Existing  CBE  staff  plus  additional  computer 

programmers  as  scheduled,  and  with  the  assistance  of  the 
CAS  during  initial  system  operation. 


This  functional  objective  is  the  ultimate  objective  of  the  entire  project, 
namely  the  operation  of  the  CE  system.  Generally,  the  CBE  personnel  involved' 
with  system  design  and  development  will  gradually  phase  the  activities  outlined 
under  this  objective  into  operation.  The  early  effort  focuses  on  getting  the 
software  implemented  onto  the  Vendor's  computer  system  and  making  sure  that 
all  subsystems  operate  properly.  Attention  then  turns  to  making  the  system 
function  instructinally  with  the  implementation  of  the  CAI/CMI  courses  and 
students. 
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ENABLING  TASK  NO.:  4.1 


PRIORITY:  1 


TASK  STATEMENT:  Collect  student  completion  times  as  individualized 

materials  are  validated,  for  the  purpose  of  building  a  data 
base  for  the  student  prediction  system. 


BEGIN:  As  early  as  October  1979  COMPLETE:  Throughout  materials  development 
PERFORMING  ORGANIZATION  ELEMENT:  CBE  operations  staff 
REQUIRED  EXPERTISE:  Level  A 


A  format  should  be  developed  for  gathering  student  completion  times  from 
the  instructional  developers  as  they  validate  instructional  materials  beginning 
with  the  CCMC  course  in  task  6.1.6.  These  student  study  times  should  also  be 
collected  along  with  the  students'  aptitude  test  scores,  so  they  may  be  placed 
in  the  proper  computer  file  when  it  is  operational.  These  items  and  aptitude 
scores  will  serve  as  the  basis  for  a  data  base  for  generating  predicted  study 
completion  times  when  the  entire  course  is  operational.  The  validation  times 
should  be  collected  for  each  instructional  module. 


ENABLING  TASK  NO. :  4.2 


PRIORITY:  1 


TASK  STATEMENT:  Staff  the  CBE  operations  group  fully  with  all  analysts  and 
progranmers. 


BEGIN:  January  1980  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  Staff 
REQUIRED  EXPERTISE:  Level  B 


At  this  time  all  operations  staff  should  be  brought  aboard.  This  step 
is  necessary  if  the  computer  systems  are  to  be  installed  in  the  vendor's 
equipment  and  operational  by  August  1980. 
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ENABLING  TASK  NO.:  4.3 


PRIORITY:  1 


TASK  STATEMENT:  Train  all  CBE  system  operators  about  operation  of 
vendor's  system. 


BEGIN:  February  1980  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  office  • 
REQUIRED  EXPERTISE:  Level  B 


The  vendor  is  to  provide  training  to  all  CBE  system  operators  on  their 
system's  operation.  This  training  must  be  completed  before  applications 
programs  can  be  transferred  to  the  new  computer  system. 
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ENABLING  TASK  NO. :  4.4 


PRIORITY:  1 


TASK  STATEMENT:  Install  applications  programs  onto  new  computer  system. 


BEGIN:  February  1980  COMPLETE:  April  1980 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  System  Operators 
REQUIRED  EXPERTISE:  Level  B 


This  work  is  required  to  transfer  application  programs  onto  the  new  system 
by  the  system  operators.  It  should  be  anticipated  that  some  programs  will 
require  change  and  debugging  during  this  transfer. 
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ENABLING  TASK  NO.: 


4.5 


PRIORITY:  1 


TASK  STATEMENT:  Begin  actual  operation  of  vendor's  computer  system  with 
system  operators  testing  out  various  programs. 

BEGIN:  April  1980  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  System  Operators 
REQUIRED  EXPERTISE:  Level  8 


As  the  subsystems  are  installed  on  the  system,  they  can  be 
independently  tested.  Nevertheless,  testing  of  the  entire  system 
cannot  occur  until  all  systems  are  online  and  operating. 
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ENABLING  TASK  NO.:  4.6 


PRIORITY:  1 


TASK  STATEMENT:  Course  authors  begin  coding  CCMC  instructional  material 
onto  the  CAI  system. 

BEGIN:  March  1980  COMPLETE:  July  1980 
PERFORMING  ORGANIZATION  ELEMENT:  Instructional  Developers 
REQUIRED  EXPERTISE:  Level  B 


To  make  sure  that  the  CAI  authoring  capability  is  truly  functional, 
instructional  developers  should  begin  coding  materials  into  the  system 
using  the  six  student  terminals.  This  coding  task  creates  the  opportunity 
for  further  testing  of  the  system  with  online  instruction.  CBE  system 
operators  and  instructional  developers  will  be  working  closely  during  this  phase 
until  procedures  are  finalized  for  course  development  and  implementation. 


ENABLING  TASK  NO.: 


4.7 


PRIORITY:  1 


TASK  STATEMENT:  Establish  administrative  procedures  for  system  operators' 
interaction  with  other  system  users,  such  as  instructional 
developers  and  training  course  personnel. 


BEGIN:  June  1980  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  Operations  Staff 
REQUIRED  EXPERTISE:  Level  A 


This  task  is  necessary  to  enable  members  of  the  training  community, 
including  instructional  developers,  instructors,  and  training  officers 
to  interface  with  the  CBE  system.  Proper  procedures  at  this  time  will 
aid  in  obtaining  acceptance  of  the  operational  system  and  will  help  to 
identify  possible  areas  of  procedural  problems. 
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ENABLING  TASK  NO.: 


4.8 


PRIORITY:  1 


TASK  STATEMENT:  Complete  site  preparations  for  CCMC  course  and  all 
system  computer  components. 

BEGIN:  July  1980  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  Operations  Staff 
REQUIRED  EXPERTISE:  Level  A 


If  the  CCMC  course  is  to  be  operational,  complete  with  students  in  August 
of  1900,  the  learning  centers  and  computer  terminals  must  be  fully  installed 
during  July.  This  will  allow  one  month  for  removing  system  problems  from 
the  terminal  to  the  computer.  It  will  also  allow  an  opportunity  for  the  CCMC 
personnel  to  become  adjusted  to  the  CBE  system.  CCMC  staff  should  not  be  ex¬ 
pected  to  teach  conventional  lectures  with  construction  going  on  around  them. 
System  acceptance  would  be  more  likely,  if  the  CCMC  course  were  not  operating 
during  July,  and  if  the  staff  were  available  to  assist  with  restructuring  of 
the  classrooms  into  CBE  learning  centers. 
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ENABLING  TASK  NO.:  4.9 


PRIORITY:  1 


TASK  STATEMENT:  Operate  the  CBE  system  with  the  CCMC  course  and  students. 


BEGIN:  August  1980  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  Operations  Staff 

REQUIRED  EXPERTISE:  Level  B 


This  task  is  what  this  entire  effort  is  about.  What  more  can  be 
said  than  what  is  expressed  in  the  task  statement.  Good  luck! 
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ENABLING  TASK  NO.:  4.10  PRIORITY:  3 

TASK  STATEMENT:  Maintain  the  CBE  system  with  courses  and  students  online. 

BEGIN:  August  1980  COMPLETE:  Through  life  of  the  system. 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  operations  staff  and  vendor  maintenance 

personnel . 

REQUIRED  EXPERTISE:  Level  B,  D 


ESTIMATED  COST: 

$72,000  for  FY  1981,  including 
maintenance  and  other  miscellane¬ 
ous  services. 

Maintenance  of  the  CBE  system  is  absolutely  vital  to  system  success.  If 
the  system  is  not  operational,  all  benefits  will  be  lost.  Even  short  down¬ 
times  are  quite  disruptive  to  students  engaged  in  CBE.  Although  it  is  difficult  to 
estimate  actual  maintenance  costs  before  system  selection,  we  feel  that  the 
MCCES  estimates  for  maintenance  are  probably  too  low.  The  CBE  operations 
staff  needs  to  monitor  maintenance  service  and  costs  and  make  sure  that  they 
are  adequate  to  keep  the  system  functional . 
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FUNCTIONAL  08JECTIVE:  Acquire  the  CBE  System 

ENABLING  TASK  NO.:  5.0  (5.1  to  5.10)  PRIORITY:  1 

TASK  STATEMENT:  Vendor  provides  equipment,  software,  and  training 
as  contracted. 


BEGIN:  June  1979  (a  2-month  setback  COMPLETE: 

caused  by  formal  delay  of 
proposal  submission.) 

PERFORMING  ORGANIZATION  ELEMENT:  Vendors 


REQUIRED  EXPERTISE:  Level  D 


Since  the  steps  in  this  functional  objective  are  already  specified 
and  dated  per  the  procurement  schedule,  these  enabling  tasks  are  duplicates 
of  those  included  on  the  implementation  plan  developed  by  the  CBE  office. 
Indicated  dates  are  scheduled  dates  plus  two  months  to  allow  for  the 
2-month  setback  in  proposal  submission. 
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5.0  Acquire  CBE  System 


Enabling  Task 

Revised  Date 

5.1 

Receipt  of  vendor  proposals  Technical  Evaluation 

June  1979 

5.2 

Benchmark  trips  to  vendors 

August  1979 

5.3 

Benchmark  Evaluation 

September  1979 

5.4 

Contract  negotiations 

October  1979 

5.5 

Best  and  Final  Offers 

October  1979 

5.6 

Selection  of  contractor  and  award  of  contract 

November  1979 

5.7 

Vendor  training  of  MCCES  personnel 

December  1979 

5.7.1 

Vendor's  training  of  Systems  Analyst  (6  weeks) 

December  1979 

5.7.2 

Vendor's  training  of  System  Operators 

February  1980 

5.8 

Vendor  installs  system,  4  management  terminals,  and 

6  student  terminals 

February  1980 

5.9 

Vendor  installs  2  optical  mark  readers  and  125 
student  terminals 

August  1980 

5.10 

Vendor  installs  2  optical  mark  readers  and  116 
student  terminals 

July  1981 
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FUNCTIONAL  OBJECTIVE:  Develop  Instructional  Materials 
Develop  Instructional  Materials 


6.0 


The  purpose  of  the  work  for  this  objective  is  to  develop  the  instruc¬ 
tional  materials  to  be  used  on  the  CBE  system  for  both  the  CAI  and  CMI  modes. 
Materials  for  the  CAI  mode  include:  the  individualized  written  materials 
of  a  narrative,  summary,  or  programmed  instruction  (PI)  nature,  along  with 
required  illustrations,  tables,  or  alternate  media.  It  is  beyond  the  scope 
of  this  analysis  to  specify  in  great  detail  the  kinds  of  material  for  each 
course.  The  materials  should  be  developed  following  a  systematic  form  of 
instructional  development  beginning  with  the  preparation  of  training  objectives 
with  appropriate  conditions  and  standards  for  performance.  For  the  CBE 
system  implementation,  extensive  job  and  training  task  analyses  will  not  be 
possible  due  to  the  time  factor;  however,  every  opportunity  should  be  taken 
to  modify  the  program  of  instruction  (POI)  for  each  course  by  incorporating 
any  needed  changes  that  are  currently  recognized  by  the  training  staff. 

These  materials  will  provide  a  hard  copy  backup  set  of  training  materials 
which  can  be  used  if  the  computer  system  becomes  temporarily  inoperative. 

The  materials  for  the  courses  to  be  implemented  only  in  a  CMI  mode  for 
initial  system  implementation  include  only  the  typical  weekly  achievement 
test.  Initial  demonstration  of  the  CMI  capability  will  require  only  student 
registration,  automated  achievement  test  scoring,  and  student  tracking  capa¬ 
bility.  The  development  and  implementation  of  the  CMI  materials  is  a  con¬ 
siderably  smaller  task  than  for  CAI,  but  it  is  important  for  ensuring  develop¬ 
ment  of  the  CMI  aspect  of  the  system.  There  are  over  40  courses  for  full  CMI 
implementation,  so  two  man-years  effort  are  devoted  to  this  task  for  each 
of  four  years.  It  is  not  possible  to  estimate  with  any  greater  degree  of 
accuracy  the  time  required  for  the  CMI  materials  development  effort  until  the 
specific  system  has  been  contractually  selected. 

A  crucial  issue  for  life-cycle  system  cost-effectiveness  regards  the 
selection  of  courses  for  CAI/CMI  implementation.  It  was  decided  that  course 
selection  should  be  based  on  student  load  factors  that  would  provide  ranking 
factors  based  on  the  number  of  student  courseweeks  taught  per  year  and  total 
annual  students/coursework  under  current  instructional  procedures.  The  follow¬ 
ing  table  shows  the  summary  of  this  analysis  with  the  courses  listed  in 
descending  order  of  student-courseweeks,  defined  as  the  total  number  of 
student-weeks  taught  each  year.  This  figure  is  obtained  by  multiplying  the 
annual  student  load  by  the  number  of  current  courseweeks.  The  second  ranking 
factor,  annual  student/coursework,  was  obtained  by  dividing  the  total  annual 
student  load  by  the  course  length  in  weeks.  The  figures  were  obtained  from 
MCCES  personnel  and  the  MCCES  Formal  School  Schedule.  The  ranking  factors 
indicate  how  much  training  occurs,  based  on  course  length.  The  higher  the 
number, the  greater  the  potential  cost-savings. 
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Annual  Student  Course-Weeks  for  MCCES  Courses  Considered  for  CBE 


Course 

Length  (wks) 

Class  Size 

Classes/yr 

Ranking  Factors 

Annual  Student  Annual  Student 
Times  Course-  Per  Courseweek 
week 

1. 

BEC 

10 

35 

48 

16,800 

168 

2. 

FROC 

7 

35 

50 

12,250 

250 

3. 

GRRC 

21 

22 

19 

8,778 

19/9 

4. 

CCMC 

9 

32 

25 

7,200 

88.9 

5. 

RF 

6 

24 

24 

3,456 

96 

6. 

FDLC 

2 

30 

48 

3,360 

840 

7. 

TTC 

16 

35 

5 

2,800 

10.9 

8. 

CCCC 

10 

25 

3 

750 

7.5 

9. 

HFCCOC 

4 

13 

9 

468 

29.2 

Because  the  annual  student-courseweek  and  total  annual  students  per  course 
week  figures  diminish  so  greatly  beyond  course  6,  FDLC,  the  remaining  courses 
(7-9)  were  not  considered  for  CAI  implementation.  There  is  just  not  a  suffi¬ 
cient  number  of  students  per  year  going  through  those  courses  to  make  them 
worth  the  extensive  instructional  development  effort  necessary  to  implement 
them  onto  the  system.  These  figures  were  also  used  as  the  basis 
for  recomnending  the  order  of  course  implementation,  except  that  the  MCCES 
indicated  that  initial  implementation  should  include  both  a  maintenance  and 
an  operator  course  in  order  to  optimize  student  training  time  savings.  For 
these  reasons  the  CCMC  course  was  selected  as  the  operator  course  because  it 
is  a  course  in  which  graduates  go  directly  to  the  field  rather  than  being 
continued  into  additional  courses  which  could  wash  away  obtained  training 
time  savings  by  having  students  wait  for  classes  to  form.  FDLC  was  selected 
as  the  maintenance  course  for  initial  implementation  because  it  is  short  and 
provides  large  numbers  of  graduates  after  only  two  weeks  training.  The  order 
of  implementation  of  remaining  courses  was  determined  on  the  basis  of  the 
amount  of  instructional  development  work  necessary,  student  load,  and  whether 
the  course  would  facilitate  movement  of  the  graduating  student  to  the  FMF, 
thereby  capturing  real  training  time  savings.  The  following  table  indicates 
the  recomnended  CAI  course  implementation  schedule. 


A- 82 


" — -  ^  ^  7  *  ,  " 


.ywnw  T^PP  i 


i'wikmw  «i.pn.r»,'«r 


Reconmended  CAI  Course  Implementation  Schedule 


1. 

CCMC 

-  June  1980 

4. 

FR0C 

-  June  1982 

2. 

FDLC 

-  October  1980 

5. 

RFC 

-  July  1983 

3. 

BEC 

-  July  1981 

6. 

FRRC 

-  June  1984 

It  should  be  noted  that  courses  were  not  excluded  even  though  they  in¬ 
volved  relatively  large  amounts  of  field-practical  work.  The  course  involved, 
primarily  FR0C,were  determined  to  be  of  such  a  nature  that  field  sites  could 
and  should  be  set  up  and  dedicated  for  particular  exercises  and  that  students 
could  go  through  those  exercises  on  an  individualized  basis  using  CMI  for 
exercise  assignment  and  monitoring.  This  would  require  staffing  those  sites 
and  would  require  the  preparation  of  written  instructions  to  aid  the  students 
in  performing  the  field  exercises,  but  would  allow  self-pacing  to  continue  to 
function.  It  might  be  appropriate  for  students  to  team  up  for  certain  exercises, 
but  this  peer-learning  experience  would  yield  important  learning  and  motiva¬ 
tional  benefits  and  would  aid  the  overall  individualized  learning  experience. 

To  aid  in  course  selection  and  in  development  of  the  implementation  plan, 
the  amount  of  instructional  development  effort  in  military  man-years  was  com¬ 
puted.  For  comparison,  the  number  of  civilian  man-years  required  was  also 
estimated.  These  estimated  man-years  were  converted  to  dollar  costs  to  aid 
in  planning.  The  basis  for  estimating  military  man-year  cost  was  the  average 
instructor  salary  of  $12,704  per  year,  while  the  estimate  for  a  civilian 
contractor  man-year  was  $37,500. 

An  additional  estimate  of  the  amount  of  instructional  development  time  per 
hour  of  instructional  time  was  necessary  for  determining  development  costs. 

This  estimate  was  the  number  of  authoring  and  coding  hours  necessary  to  con¬ 
vert  each  hour  of  conventional  instruction  to  CAI  or  CMI.  For  purposes  of 
the  recommended  implementation  plan  and  for  the  sensitivity  analyses,  high, 
moderate,  and  low  estimates  for  required  time  were  selected  and  used.  These 
authoring/coding  estimates  are  presented  in  the  table  below. 


Selected  Authoring/Coding  Hour  Estimates  Per  Instructional  Hour 

Low  Moderate  High 

Author  Code  Author  Code  Author  Code 


CAI 

75 

25 

100 

50 

150 

75 

CMI 

75 

10 

100 

20 

150 

30 

The  moderate  time  estimates  were  used  in  determining  the  required  develop 
ment  timesand  costs, which  are  included  in  the  recommended  implementation  plan. 
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A  further  important  assumption  was  that  all  practical  or  exercise  POI  hours 
would  be  taught  using  CMI,  and  that  50%  of  the  knowledge  portion  of  each 
course  would  be  developed  for  CAI  and  the  remaining  50%  developed  for  CMI. 
The  decision  to  base  CAI  development  time  for  knowledge  portions  of  courses 
on  a  50%  ratio  was  arbitrary,  but  was  predicated  on  the  decision  to  include 
six  courses  for  implementation  instead  of  four,  thereby  resulting  in  a  need 
for  more  student  terminals  than  an  85%  CAI  ratio  would  permit.  Even  though 
contractual  specifications  limit  the  acquisition  to  260  student  terminals, 
that  may  well  be  an  adequate  njnber  for  all  six  courses  given  this  report's 
reconmendation  of  only  50%  CAI  for  knowledge  POI  hours. 


The  following  tabel  summarizes  POI  hours,  and  estimated  required  man- 
years  and  costs  for  both  civilian  and  military  instructional  developers. 

A  military  manager  was  figured  on  1,693  hours  per  year  (MCCES  hours  figure 
of  1765-72  hours  for  holidays)  and  1,920  hours  per  year  for  civilian  (40 
hours  per  week  times  48  weeks). 


Course  POI  Hours  and  Estimated  Man-years  and  Costs  for  Military  and  Civilian 
Contractor  Instructional  Development 


Course 

POI  Hours 

Military 
Man-years  Costs 

Contractor 
Man-years  Costs 

1. 

CCMC 

333 

24.5 

311. IK 

21.7 

813. 7K 

2. 

FDLC 

70 

5.3- 

67. 3K 

4.7 

176. 3K 

3. 

BEC 

359 

27.6 

350. 5K 

24.4 

915K 

4. 

FROC 

279 

20.6 

261. 6K 

18.2 

682. 5K 

5. 

RFC 

222 

17.1 

217. 2K 

15.1 

566. 3K 

6. 

GRRC 

704 

53.5 

679. 5K 

47.3 

1,773. 7K 

It  should  be  apparent  from  the  figures  in  the  table  that  development  of 
materials  in  general  is  extremely  expensive  and  development  of  the  GRRC  course 
will-be  the  most  expensive  of  all.  It  was  included  because  it  does 
have  a  high  student  annual  throughout  and  will  contribute  to  the  overall 
cost-effectiveness  of  the  CBE  system.  Obviously,  as  instructional  develop¬ 
ment  resources  are  limited,  GRRC  should  be  the  last  course  to  be  developed. 

Development  of  the  materials  is  one  of  the  most  important  activities 
ancMs  crucial  to  the  success  of  the  system.  Well -developed  materials  will 
do  more  for  system  success  than  any  other  area  of  endeavor  except  for  system 
design.  Therefore,  it  is  our  reconmendation  that  instructional  materials 
be  developed  by  contracted  professional  material  developers.  Recognizing 
that  funds  may  not  be  available  for  all  contractor  developed  materials,  we 
recommend  that  the  first  two  courses  to  be  developed,  CCMC  and  FDLC,  be 
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done  under  contract  and  then  those  materials  can  be  used  by  Marine  Corps 
instructional  developers  as  models  for  the  construction  of  materials  for 
other  courses.  In  the  interest  of  economy,  the  man-year  and  cost  estimates 
for  military  development  were  included  in  the  recommended  implementation  plan, 
since  it  was  understood  at  the  beginning  of  this  analysis  that  MCCES  was 
arranging  for  32  instructor  billets  for  personnel  to  serve  for  four  years 
to  prepare  materials. 

If  the  necessary  funds  for  contractors  or  the  military  billets  for 
developers  are  not  provided,  the  CBE  system  will  NEVER  become  operational. 

To  the  extent  that  the  funds  or  billets  are  only  partially  provided,  then 
only  a  portion  of  the  courses  can  be  implemented.  Training  time  savings, 
hence,  cost-effectiveness,  can  be  obtained  only  by  developing  and  implementing 
individualized  instruction  on  the  CBE  system.  The  importance  of  the  material 
development  task  and  the  development  resource  requirement  can  not  be  under¬ 
estimated. 

Although  not  a  major  concern  of  MCCES,  it  is  common  knowledge  that  all 
branches  of  the  military,  including  the  Marine  Corps  are  experiencing  man¬ 
power  recruitment  problems.  The  potential  impact  of  professionally  developed 
instructional  materials  on  student  morale  may  be  expected  to  extend  into  the 
civilian  community;  recruits  telling  their  friends  that  their  courses  are 
interesting  and  exciting  could  do  more  for  the  Corps  image  than  any  number 
of  recruiter  pep  talks  or  expensive  TV  commercials. 

If  a  contractor  is  to  be  used  for  development  of  materials,  the  contract 
should  be  let  on  a  competitive  basis.  Without  competition,  the  price  for 
contractor  developed  materials  would  undoubtedly  be  higher  than  necessary 
for  obtaining  suitable  materials. 

The  information  in  the  preceding  paragraphs  has  been  provided  for  personnel 
using  the  implementation  to  enable  them  to  perform  the  necessary  calcula¬ 
tions  for  an  instructional  development  cost  analysis  should  situational  con¬ 
straints  call  for  deviation  from  the  recommended  course  development  list.  The 
enabling  tasks  that  are  necessary  for  meeting  the  instructional  materials 
objective  are  specified  in  the  following  enabling  task  summary  sheets. 


FUNCTIONAL  OBJECTIVE:  Develop  Instructional  Materials 


ENABLING  TASK  NO.:  6-0  PRIORITY:  1 

TASK  STATEMENT:  Develop  individualized  instructional  materials  for  CBE 
system. 


BEGIN:  August  1979  COMPLETE:  When  system  fully  implemented. 

PERFORMING  ORGANIZATION  ELEMENT:  Preferred:  contract  materials  developer, 

but  MCCES  personnel  if  necessary. 

REQUIRED  EXPERTISE:  Level  D  or  A:  Instructional  material  development. 
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ENABLING  TASK  NO.  :  6.1  PRIORITY:  ) 

TASK  STATEMENT:  Develop  CAI  CMI  instructional  material  for  Communication 
Center  Main  Course  (CCMC) 


BEGIN:  August  1979  COMPLETE:  June  1980 


PERFORMING  ORGANIZATION  ELEMENT:  ESD  instructional  developers 


REQUIRED  EXPERTISE:  Level  B,  but  prefer  D:  E-6  Instructor  with  knowledge  of 

subject  matter  area  and  capable  of  performing  ISD  pro¬ 
cedures.  Ideally  this  task  would  be  performed  by  a 
highly  skilled  instructional  developer  under  contract 
to  MCCES.  Expected  civilian  cost  =  21.7  civilian  man- 
years/$813.7K. 


ESTIMATED  MILITARY  MAN-YEARS:  24.5  ESTIMATED  COST:  $311. IK 

Enabling  tasks  for  development  of  the  first  course  for  CBE  implementation 
include  initial  tasks  which  define-indi vidual ized  materials  format  and  author 
training.  The  development  for  subsequent  courses  does  not  include  those  activities. 
Separate  manning/costs  for  each  subtask  are  not  desirable  without  extensive  ISD 
planning  effort  and  are  beyond  the  scope  of  this  analysis. 
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ENABLING  TASK  HO.:  6.1.1 


PRIORITY:  1 


TASK  STATEMENT:  Define  format  and  procedures  for  developing  individualized 
materials  for  CAI/CMI. 


BEGIN:  June  1979  COMPLETE:  July  1979 


PERFORMING  ORGANIZATION  ELEMENT:  ESD  Educational  Specialist 


REQUIRED  EXPERTISE:  Level  A,  or  D  Must  have  knowlege  of  ISD  procedures, 

and  be  familiar  with  individualized  materials. 


This  task  specifies  the  format  for  the  CAI  materials,  building  upon 
experience  with  programmed  instruction.  General  constraints  on  media,  subject 
matter  experts,  and  materials  must  be  taken  into  consideration  when  specifying 
the  format.  Although  exact  instructional  sequencing  strategies  cannot  be 
specified  until  contract  award,  general  course  sequencing  strategies  can  be 
defined.  Arrangements  should  be  made  with  Centers  graphics  department  for 
providing  graphics  support. 
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ENABLING  TASK  NO. :  6.1.2 


PRIORITY:  l 


TASK  STATEMENT:  Train  M.C.  personnel  in  preparation  of  CAI  materials 

BEGIN:  July  1979  COMPLETE:  August  1979 

PERFORMING  ORGANIZATION  ELEMENT:  ESD  personnel 
REQUIRED  EXPERTISE:  Level  A 


ESD  personnel  will  train  E-6  instructors  who  have  been  selected  for 
instructional  development  effort.  They  should  be  taught  how  to  apply 
ISD  procedures  including:  writing  of  training  objectives,  writing  of 
test  items,  specification  of  learning  activities,  and  preparation  of 
narrative,  programmed  instruction,  and  summaries  of  technical  instructional 
material . 


A- 89 


ENABLING  TASK  MO. :  6.1.3 


PRIORITY:  1 


TASK  STATEMENT:  Conduct  anal  sis  of  POI  for  CCMC 


j 

BEGIN:  COMPLETE:  j 


PERFORMING  ORGANIZATION  ELEMENT:  MCCES  instructional  developers 

REQUIRED  EXPERTISE:  Level  B 


This  analysis  should  include  examination  of  current  training  objectives 
in  the  CCMC  POI,  and  should  integrate  identified  curriculum  changes  provided 
by  the  CCMC  training  staff  from  Task  1.2.1.  Upon  revision  of  objectives, 
learning  activities  should  be  specified,  and  test-items  which  measure 
achievement  of  the  objectives  should  be  written. 


( 


ENABLING  TASK  NO.: 


6.1.4 


PRIORITY:  l 


TASK  STATEMENT:  Prepare  individualized  instructional  materials  for  CCMC. 

BEGIN:  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  Instructional  developers 

REQUIRED  EXPERTISE:  Level  B 


Instructional  materials  should  be  written  in  both  narrative  and  programmed 
instruction  formats  in  order  to  suit  different  learning  styles  of  students.  A 
summary  of  material  should  also  be  prepared  for  each  module. 


ENABLING  TASK  NO.:  6.1.5 


PRIORITY: 


1 


TASK  STATEMENT:  Develop  graphics  material  for  CCMC 


BEGIN:  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  Base  graphics'  department 
REQUIRED  EXPERTISE:  Level  A 


Graphics  material  should  be  provided  back  to  instructional  developers 
as  rapidly  as  possible  to  facilitate  review  and  revision,  if  necessary. 

It  would  be  expected  that  most  graphics  work  would  require  only  black  and 
white  written  materials. 


ENABLING  TASK  NO.: 


6.1.6 


PRIORITY: 


1 


TASK  STATEMENT:  Validate  individualized  materials  for  CCMC 

BEGIN:  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  Instructiona-T  Developers/ CCMC  staff 
REQUIRED  EXPERTISE:  Level  B 


Validate  instruction  by  using  approximately  6-10  students  from  current 
CCMC  class  students  who  have  not  experienced  training  in  relevant  objectives. 
This  CCMC  input  comes  from  Task  1.3.1  and  should  involve  individual  students 
for  only  a  day  or  two  at  a  time.  The  time  that  these  students  require  for 
completing  these  materials  should  be  measured  and  reported  to  the  CBE  office 
to  aid  in  developing  the  student  progress  prediction  capability  of  the  system 
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ENABLING  TASK  NC. :  6.1.7  PRIORITY:  1 

TASK  STATEMENT:  Revise  CCMC  instructional  materials  as  required. 

BEGIN:  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  Instructional  Developers 

REQUIRED  EXPERTISE:  Level  B 


Any  necessary  revisions  in  written, 
or  graphics  work  should  be  performed  at 
copies  of  completed  materials  should  be 
course  without  CAI--computer  assistance 


programmed,  instructional  materials 
this  time.  A  sufficient  number  of 
reproduced  for  initial  operation  of 
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ENABLING  TASK  NO.:  6.1.8 


PRIORITY:  l 


TASK  STATEMENT:  Train  instructional  developers  in  Vendor's  Author 
Language 


BEGIN:  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  Education  specialists  from  ESD 

REQUIRED  EXPERTISE:  Level  A 


This  step  assumes  that  education  specialists  from  the  ESD  department 
have  themselves  received  the  appropriate  training  from  the  Vendor  repre¬ 
sentatives,  so  that  the  MCCES  personnel  can  provide  additional  training 
directly  to  the  MC  developers. 


ENABLING  TASK  NO.  :  6.1.9  PRIORITY:  -| 

TASK  STATEMENT:  Sequence  individualized  instructional  materials  according 
to  the  strategies  specified  by  the  Vendor  Author  Language. 


BEGIN:  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  Instructional  Developers 

REQUIRED  EXPERTISE:  Level  B 


Sequencing  of  the  CCMC  materials  according  to  the  Author  language  may 
take  more  time  than  for  subsequent  courses  since  it  will  be  the  first 
occasion  this  task  is  undertaken.  Therefore,  this  sequencing  may  result 
in  some  additional  material  revision.  Management  of  practical  exercises 
must  be  considered  in  the  development  of  an  instructional  sequence.  Time  must 
be  allowed  for  student  relocation  from  learning  center  to  field  instruc¬ 
tional  sites. 
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ENABLING  TASK  NO.: 


6.1.10 


PRIORITY: 


1 


TASK  STATEMENT:  Code  CCMC  materials  onto  CAI  system 


BEGIN:  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  Instructional  developers 

REQUIRED  EXPERTISE:  Level  B 


The  developers  must  have  received  the  Vendor  Author  training  in  order 
to  code  the  materials  onto  the  system.  Since  this  is  the  first  course  to 
be  implemented,  this  task  may  require  some  revision  in  the  materials  for 
success.  Coding  must  include  allowance  for  necessary  change  from  CAI  to 
CMI  mode. 


ENABLING  TASK  NO.: 


6.1.11 


PRIORITY: 


1 


TASK  STATEMENT:  Validate  CAI  materials  with  CCMC  students 


BEGIN:  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  Instructional  developers 

REQUIRED  EXPERTISE:  Level  B 


Validation  of  materials  will  require  that  the  system  be 
a  small  sample  of  students,  approximately  6-10.  CCMC  staff 
students  and  their  perceptions  regarding  material  from  Task 
should  also  include  verifying  the  material  s' ability  to  move 
from  CAI  to  CMI  modes  and  back. 


operational  for 
will  input 
1.4.1.  Validation 
students 


ENABLING  TASK  NO.: 


6.1.12 


PRIOR  Hr: 


1 


TASK  STATEMENT:  Revise  CAI  materials  as  required. 


BEGIN:  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  Instructional  developers 
REQUIRED  EXPERTISE:  Level  B 


Course  authors  must  revise  the  CAI  materials  based  on  student  performance. 
This  task  must  be  done  for  this  first  course,  but  then  it  will  be  done  as  a 
scheduled  course  revision  procedure.  Course  revision  should  occur  approximately 
every  12-24  months,  depending  upon  available  resources.  The  procedures  speci¬ 
fied  in  the  Ford  Aerospace  report  of  16  November  1978  may  be  used  as  a  starting 
point  for  developing  a  decision  analysis  process  for  selecting  a  revision 
strategy. 


ENABLING  TASK  NO.: 


6.1 .13 


PRIORI!  ;  : 


1 


TASK  STATEMENT:  Turn  CCMC  course  materials  over  to  school 


BEGIN:  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  Instructional  developers 

REQUIRED  EXPERTISE:  Level  B 


CAI  materials  should  be  turned  over  to  CCMC  staff,  via  the  CBE 
operations  personnel.  Additionally,  backup,  hard-copy  programmed  instruction 
should  also  be  delivered  to  the  CCMC  in  order  to  provide  the  staff  with 
materials  in  case  the  computer  malfunctions.  Assistance  should  be  provided 
to  the  staff  to  aiding  the  preparation  of  field  sites  for  particular  exercises  to  be 
monitored  by  CMI. 
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ENABLING  TASK  NO.:  6.2  PRIORITY:  ■, 

TASK  STATEMENT:  Develop  instructional  materials  for  FDLC 


BEGIN:  June  ^80  COMPLETE:  October  1980 


PERFORMING  ORGANIZATION  ELEMENT:  Instructional  developers;  prefer  contractors, 

but  plan  specifies  actions  for  military 
developers. 

REQUIRED  EXPERTISE:  Level  D  preferred,  but  task  can  be  done  by  Level  B. 


ESTIMATED  MAN  YEARS:  5.3  fflY 


The  procedures  followed  in  this  course  development  effort  should  be 
repeated  for  all  remaining  courses  to  be  implemented  on  the  CAI/CMI  system. 


ENABLING  TASK  NO.: 


6.2.1 


PRIORI! < : 


1 


TASK  STATEMENT:  Train  personnel  in  preparation  of  instructional  materials 
and  Vendor  Author  language,  as  required. 


BEGIN:  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  Instructional  developers 
REQUIRED  EXPERTISE:  Level  B 


If  new  personnel  are  brought  in  for  this  development  effort,  then  they 
must  receive  the  training  in  materials  development  and  course  authoring 
presented  to  earlier  developers.  Probably,  some  of  these  individuals  will 
have  worked  on  the  CCMC  course-,  they  may  serve  as  "Team  Leaders"  which 
will  enable  the  training  course  objectives  to  be  divided  into  sections  so  that 
simultaneous  development  may  be  carried  out  by  several  teams. 
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ENABLING  TASK  NO.: 


6.2.2 


PRIORITY  1 


TASK  STATEMENT:  Conduct  analysis  of  the  FDLC  POI  and  staff  input  regarding 

required  changes  in  the  course  curriculum. 

BEGIN:  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT :  Instructional  developers 
REQUIRED  EXPERTISE:  Level  B 


This  analysis  is  identical  to  the  analysis  performed  in  Task  6.1.3 
where  school  staff  provide  training  change  input  via  Task  1.2.2.  This 
work  is  critical  to  the  creation  of  a  course  that  is  directed  at  the  most 
recent  information  regarding  training  objectives. 


ENABLING  TASK  NO.:  6.2.3  Phi C » I T  f :  1 

TASK  STATEMENT:  Develop  individualized  materials  for  FDLC  and  sequence 

according  to  Vendor  strategy  for  instructional  presentation. 

DEGIN:  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  Instructional  developers 
REQUIRED  EXPERTISE:  Level  B 


Development  of  these  materials  may  be  different  from  the  CCMC  course, 
since  FDLC  contains  more  laboratory  work.  Management  strategies  for 
monitoring  students  in  the  laboratory  sessions  must  also  be  developed. 
Care  must  be  taken  to  develop  proper  student  instructions  to, insure  that 
students  can  proceed  with  laboratory  exe-Mses  with  minimal  instructor 
assistance. 
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ENABLING  1  ASK  NO. : 


6.2.4 


1 


TASK  STATEMENT :  Develop  graphics  materials  for  FDLC 

BEGIN: 

PE  WORMING  ORGANIZATION  LlLMENT:  Base  graphic’s  department 

REQUIRED  EXPERTISE:  Level  A 


Development  of  graphics  materials  should  be  relatively  routine  at  this 
point,  since  a  previous  course  has  already  been  developed  and  administrative 
problems  presumably  will  have  been  resolved. 
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LNABLING  TASK,  NO.: 


6.2.5 


PRIORITY: 


1 


TASK  STATEMENT:  Validate  individualized  materials  for  FDLC 

BEGIN:  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  Instructional-  developers 
REQUIRED  EXPERTISE:  Level  B 


Validation  of  written  individualized  materials  should  be  completed  using 
input  from  the  FDLC  staff  and  approximately  6-10  students,  via  Task  1.3.2. 
Any  revision  should  be  accomplished  during  this  task. 
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ENABLING  TASK  NO.:  6.2.6  PRIORITY:  1 

TASK  STATEMENT:  Code  materials  into  CAI  system  using  Vendor  Author 
Language . 

BEGIN:  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  Instructional  developers 
REQUIRED  EXPERTISE:  Level  B 


Course  coders  should  be  well  trained  at  this  point  and  the  materials 
should  be  coming  out  of  the  development  phase  closer  to  a  final  acceptable 
format. 
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ENABLING  TASK  NO.:  6.2.7 


PRIORITY: 


1 


TASK  STATEMENT:  Implement  individualized  materials  and  system  into  the 
FDLC  course. 


BEGIN:  COMPLETE: 


PERFORMING  ORGANIZATION  ELEMENT:  Instructiorlal  developers 


REQUIRED  EXPERTISE:  Level  B 


FDLC  instructional  materials  should  be  turned  over  to  course  personnel. 
Assistance  from  CBE  operations  personnel  should  be  provided  to  the  staff 
to  assist  them  in  learning  the  materials  and  CAI  system.  Additional 
assistance  should  be  made  available  to  ensure  that  the  staff  can  manage 
the  transitions  from  CAI  to  CMI  and  from  knowledge  to  laboratory  sections 
of  the  course , 


ENABLING  TASK  NO.: 


6.2.8 


PRIORITY:  I 


TASK  STATEMENT:  Revise  FDLC  individualized  materials  in  accordance  with 

normal  MCCES  policy  about  course  changes. 


BEGIN:  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  Instructional  development 
REQUIRED  EXPERTISE:  LEVEL  A 


Revision  of  materials  should  be  performed  on  a  12-24  month  basis,  dependi 
upon  resource  availability.  It  would  be  expected  that  normal  billeted  MCCES 
personnel  in  the  ESD  department  would  perform  the  revision,  relying  on  input 
from  the  course  staff  via  Task  1.5.2. 


ENABLING  TASK  NO.:  6.3  through  6.6. 


PRIORITY:  3 


TASK  STATEMENT:  Develop  the  instructional  materials  for  remaining  courses 
to  be  implemented  upon  CAI/CMI  system 


BEGIN:  COMPLETE:  Insructional  developers 

PERFORMING  ORGANIZATION  ELEMENT:  Instructional’  developers 
REQUIRED  EXPERTISE:  Level  B 


Sub-tasks  under  Enabling  Objective  6.2  should  be  repeated  for  each  additional 
course.  These  sub-tasks  include:  course  analysis,  material  development,  valida¬ 
tion,  coding,  and  turning  materials  over  to  the  course  staff.  As  each  course  is 
implemented,  procedures  will  become  more  effective  and  streamlined.  If  contrac¬ 
tor  support  has  been  provided  for  developing  the  CCMC  and  the  FDLC  courses,  then 
those  materials  should  be  used  as  models  for  development  of  other  courses  by 
Marine  Corps  personnel. 
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ENABLING  TASK  NO.: 


6.? 


PRIORITY:  3 


TASK  STATEMENT:  Develop  material  for  BEC 

BEGIN:  June  80  COMPLETE:  July  81 

PERFORMING  ORGANIZATION  ELEMENT:  Instruction<!l  developers 

REQUIRED  EXPERTISE:  Level  B 


ESTIMATED  MAN  YEARS:  27.6  MMY 
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•V. 


ENABLING  TASK  NO.: 


6.4 


PRIORITY: 


TASK  STATEMENT:  Develop  material  for  FROC 

BEGIN:  July  1931  COMPLETE:  June  1982 

PERFORMING  ORGANIZATION  ELEMENT:  Instructional  developers 

REQUIRED  EXPERTISE:  Level  B 


ESTIMATED  MAN  YEARS:  20.6  MW 


■i 


ENABLING  TASK,  NO. :  o.b 

TASK  STATEMENT:  Develop  naterials  for  RFC 


BEGIN:  July  1981  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  Instructional 

REQUIRED  EXPERTISE:  Level  B 
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PRIORITY: 

July  1983 
developers 


"" - p- - *»- •  TV  — ■« 

Ti'i 


I 


ENABLING  TASK  NO.:  &6  PRIORITY: 

TASK  STATEMENT:  Develop  materials  for  GRRC 

BEGIN:  June  1982  COMPLETE:  June  1984 

PERFORMING  ORGANIZATION  ELEMENT:  Instructional  developers 

REQUIRED  EXPERTISE:  Level  B 


ESTIMATED  MAN  YEARS: 


53.5 


ENABLING  TASK  NO.: 


6.7 


PRIORITY: 


1 


TASK  STATEMENT:  Development  of  CMI  materials  for  courses  to  be  implemented 

on  CMI  system  only. 


BEGIN:  October  1980  COMPLETE:  August  1984 


PERFORMING  ORGANIZATION  ELEMENT:  Instructional  developers 


REQUIRED  EXPERTISE:  Level  B 


Courses  that  are  to  remain  in  lecture  form,  but  are  to  be  implemented  on 
CMI  for  testing  and  management  should  be  selected  and  have  respective  course 
staff  provide  the  necessary  test  items  and  course  description  for  proper 
coding  into  the  CBE  system.  Instructional  developers  would  be  expected  to 
code  tests  into  the  CMI  system  with  course  staff  providing  any  requirements 
for  test  change  or  course  sequence  change.  As  development  resources  are 
made  available,  these  lock-step  courses  should  be  converted  to  full  CAI/CMI 
presentation.  This  conversion  would  not  be  expected  to  occur  prior  to  August 
1984.  Exact  estimates  of  the  level  of  effort  required  for  this  enabling  task 
cannot  be  specified  at  this  time. 


ENABLING  TASK  NO.: 


6.8 


PRIORITY: 


3 


I 

I 

TASK  STATEMENT:  Revise  course  materials  for  CBE  courses  according  to  MCCES 
'  policy 


BEGIN:  As  retired  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  ESD 

REQUIRED  EXPERTISE:  Level  A 


Revision  of  materials  should  be  conducted  according  to  standard  Marine 
Corps  policy,  approximately  every  12-24  months,  depending  upon  resource 
avai labil ity. 
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FUNCTIONAL  OBJECTIVE: 


Train  CBE  System  Instructors 


ENABLING  TASK  NO.:  7>0  PRIORITY:  j 

TASK  STATEMENT:  Train  personnel  to  assume  the  role  of  instructor  in  the 
CBE  system. 

BEGIN:  Sept  1979  COMPLETE:  Continuing 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  office  and  ESD  department. 

REQUIRED  EXPERTISE:  Level:  A  and  D  for  development  of  training  materials. 


CMI  instructor  role  training  materials  need  to  be  developed  and  implemented 
into  the  instructor  training  program  at  MCCES.  Because  of  the  new  nature  of 
this  system,  development  of  the  instructor  training  materials  should  be  done 
by  an  outside  organization  experienced  with  this  activity.  Availability  of 
a  planned  R&D  contract  with  McDonnell -Douglas  Corporation  to  develop  a  CMI 
instructor  role  model  for  the  Navy,  Air  Force  and  the  Marine  Corps  systems 
should  be  used  to  obtain  the  CMI  instructor  job  specification. 
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ENABLING  TASK  NO.  :  7.1 


PRIORITY: 


1 


TASK  STATEMENT:  -]-^e  cbe  office  should  input  into  the  McDonnell -Douglas  CMI 
Instructor  Role  R&D  project  to  make  sure  that  the  idealized 
role  model  developed  by  the  contractor  can  be  best  suited 
for  the  proposed  Marine  Corps  CBE  system. 

BEGIN:  Sept  -  Dec  1979  COMPLETE: 


PERFORMING  ORGANIZATION  ELEMENT:  CBE  office/training  personnel 


REQUIRED  EXPERTISE:  Level  A 


This  task  will  serve  to  inform  the  contractor  of  the  expected  dimensions 
of  the  MCCES  CBE  system  and  the  functional  activities  that  will  be  expected 
of  the  CBE  system  instructor. 
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ENABLING  TASK,  NO.:  7.2 


p  c  ;  (_  u ;  ; 


1 


TASK  STATEMENT:  Receive  the  general  CMI  instructor  role  model  from  McDonnel- 
Douglas  and  the  job  specification  for  the  MCCES  CBE  system. 


BEGIN:  jan  1980  (assuming  contract  COMPLETE: 
is  let  as  scheduled. ) 

PERFORMING  ORGANIZATION  ELEMENT:  CBE  off ice/ESD'department 
REQUIRED  EXPERTISE:  Level  A 


This  task  is  necessary  to  obtain  the  job  specification  and  verify  it 
against  the  CBE  system  as  it  has  been  developed.  Receipt  of  the  materials 
of  the  indicated  data  is  dependent  upon  the  contract  being  let  as  scheduled. 


A- 119 


ENABLING  TASK  NO. :  7.3 


PRIORITY:  i 


TASK  STATEMENT:  Develop  the  CMI  instructor  training  package. 


BEGIN:  Jan-June  1980 


COMPLETE: 


PERFORMING  ORGANIZATION  ELEMENT:  Contractor,  NPRDC  could  monitor. 


REQUIRED  EXPERTISE:  Level  D:  Expertise  in  developing  instructor  training 

materials,  preferably  for  computer-based  instructional 
systems. 


ESTIMATED  MAN  YEARS : 


ESTIMATED  COST-.  S35K 


This  CMI  instructor  training  material  development  effort  will  rely  upon 
obtaining  the  MCCES  CMI  Instructor  job  specification  from  the  McDonnell -Douglas 
contract.  Completion  of  training  materials  development  could  occur  approximately 
6  months  after  receipt  of  the  instructor  job  specification.  An  initial  try¬ 
out  of  the  materials  would  be  conducted  during  the  expected  6  month  development 
period. 
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LNABIING  TASK  NO. :  7.4 


PRIORITY:  1 


TASK  STATEMENT:  Train  a  sample  of  MCCES  personnel  to  the  CMI  instructor  role 
job  specification. 


BEGIN:  Training  is  expected  to  take  COMPLETE: 
one  week,  (July  1980,  if  conr 
tract  is  let  on  scedule). 

PERFORMING  ORGANIZATION  ELEMENT:  Contract  organization  that  developed  training 

materials. 

REQUIRED  EXPERTISE:  Level  D 


This  activity  would  be  part  of  the  CMI  instructor  training  materials 
development  effort.  It  would  serve  as  a  test  of  the  training  materials, but 
would  not  have  the  benefit  of  indicating  how  the  instructors  actually  performed. 
That  would  occur  once  the  CCMC  course  was  implemented. 
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ENABLING  TASK  NO.  :  7.5 


P  P I  OR  ITT:  1 


TASK  STATEMENT:  Implement  trained  instructors  into  the  CCMC  course. 


BEGIN:  August  1980 


COM PL  ETE : 


PERFORMING  ORGANIZATION  ELEMENT:  CCMC  personnel 


REQUIRED  EXPERTISE:  Level  A 


Personnel  trained  in  the  CMI  instructor  role  would  be  assigned  to  the 
new  CCMC  course  taught  by  CAI/CMI.  The  effectiveness  of  the  instructor 
training  would  be  determined  by  the  adequacy  of  the  instructors  serving  in 
CCMC.  Since  the  technical  course  is  also  new,  it  would  not  be  possible  to 
assign  all  effects  to  either  the  instructor  training,  or  the  effectiveness 
of  the  CBE  system  and  instructional  materials. 


ENABLING  TASK  NO.: 


7.6 


PRIORITY:  3 


TASK  STATEMENT:  Select  instructors  and  conduct  CMI  instructor  training  as 
required  for  new  CBE  courses. 

BEGIN:  As  required  COMPLETE: 

PERFORMING  ORGANIZATION  ELEMENT:  ESD/ instructor' training  personnel. 
REQUIRED  EXPERTISE:  Level  A 


CMI  instructor  training  should  become  a  part  of  the  normal  instructor 
training  activities  in  MCCES,  providing  instruction  for  those  individuals 
selected  for  duty  in  the  CBE  system  courses.  Although  it  is  assumed  that 
selection  procedures  for  CBE  instructors  will  be  the  same  as  those  now  used  for 
lecture  instructors  there  may  be  better  selection  methods.  Determination 
of  better  instructor  selection  methods  remains  to  be  done.  What  is  not  assumed 
is  that  there  will  be  the  same  number  of  instructors  for  each  course  under 
CME  as  are  now  assigned.  It  is  necessary  to  have  some  guidelines  for  determining 
the  CBE  instructor  requirement.  The  following  guidelines  were  used  in  determining 
the  estimated  instructor  requirement  for  the  recommended  CBE  system  and  should 
be  used  in  actual  instructor  assignment  until  better  guidelines  are  developed. 

In  general,  portions  of  courses  that  are  of  a  knowledge  acquisition  nature 
require  instructors  on  the  basis  of  30  students  per  instructor  in  a  CMI 
learning  center.  Practical  portions  of  CBE  courses  are  to  be  operated  with 
student/instructor  ratios  of  10  students  per  instructor.  This  higher  ratio 
depends  upon  wel 1 -developed  instructional  materials  to  guide  each  student  in 
the  particular  exercise  or  laboratory  experiment  he  is  working  on.  The  applica¬ 
tion  of  these  ratios  to  the  respective  proportion  of  each  course  selected  for 


CAI  or  CMI,  with  an  allowance  for  courses  having  field  exercises,  results 
in  the  following  instructor  requirements.  Current  course  instructor  require¬ 
ments,  as  provided  by  CBE  staff,  are  included  to  indicate  the  potential  savings 
in  instructor  personnel.  It  should  be  remembered  that  some  instructor  reduc¬ 
tions  are  off-set  by  increases  in  CBE  staff. 


Course 

Current  Lecture 
Instruction 

CBE 

Number 

Reduced 

CCMC 

14 

10 

4 

FDLC 

5 

3 

2 

BEC 

29 

22 

7 

FROC 

30 

22 

8 

RFC 

15 

9 

6 

GRRC 

22 

15 

7 

115 

81 

34 

A- 124 


APPENDIX  B 


LIST  OF  TECHNICAL  AND  COST  DATA  ELEMENTS 
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LIST  OF  TECHNICAL  AND 
COST  DATA  ELEMENTS 


Prior  to  visiting  MCCES  in  March  1979  the  following  list  of  technical  and 
cost  data  elements  was  developed  to  identify  types  of  information  necessary 
for  cost  estimating  purposes.  Ideally,  data  sets  for  each  of  these  elements 
would  be  available  to  support  total  life  cycle  costing  of  alternative  methods 
of  instruction.  In  reality,  limited  data  were  available  on  specific  Cl 
course  costs  for  all  the  elements  listed.  On  the  casis  of  a  one-week 
visit  to  MCEES  and  liaison  with  MCCES  personnel,  the  availability  of  infor¬ 
mation  for  each  cost  element  was  judged  to  be:  a  =  adequate;  b  =  inadequate; 
or  c  =  not  available.  Inadequate  implies  that  cost  and  technical  factors 
were  not  readily  available,  but  could  be  developed  with  varying  degrees  of 
effort  from  raw  data  which  were  accessible. 

A  list  of  cost  and  technical  elements  follows: 

Course  Number  and  Title  of  Courses  Considered  for  CBEa 

Length  of  Training  by  Course  Number  and  Title3 

Total  Cost  of  Training  by  Course  Number  and  Title*3 

Student  Load:  The  average  number  of  students  in  training  who  must  be3 
instructed,  supervised,  and  otherwise  supported. 

Student-Instructor  Ratio  (Students  per  Instructor)3 

Student  Turnover:  The  rate  of  flow  considering  variable  course  lengths; 

impacts  staffing  for  personnel  administration  and  . 
processing,  student  supervision  and  transportation. 

Attrition  Rates  by  Course0 

Student  Paygrade  by  Course*3 

Number  and  Average  Grade  of  Instructor  Personnel  by  Course*3 
Total  Twentynine  Palms  Training  Requirements3 
Instructional  System  Design  Costs*3 
Courseware  Development  Costs*3 

•  Instructional  hours  developed;  cost  per  manyear  (consider  direct  and 
indirect  costs  such  as:  personnel  costs  associated  with  course 
authoring,  coding  and  task  analyses;  supplies/consumables;  overhead, 
printing  costs,  etc.) 

•  Course  Revision  Costs 

•  Task  Analyses 
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Instructional  Equipment  Costs  (specify  quantity  and  unit  cost)0 

•  Audio/Visual  Aids 

•  Programmed  Texts 

•  Test  Materials 

•  Optical  Mark  Readers 

•  Carrels  , 

•  Other  (list  lab  equipment  used  for  hands-on  training) 

Computer  Hardware  (specify  configuration,  quantity  and  unit  cost)*3 

Computer  Software  Development  Costs*3 

Computer  Facilities  Modification  Costs  (sunk  costs)*3 

Computer  Operation  &  Maintenance  Costs*3 

t  Personnel  Costs  (system  analysts,  engineers,  programmers,  computer 
operators,  keypunch) 

•  Maintenance  Costs 

Instructor  Training  Costs  (course  costs)*3 

r 

Manpower  Staffing  Studies;  cost  of 
CBE  Supplies/Consumables;  cost  ofd 
Travel  Costs  (Student  and  Staff)*3 

CBE  Contractual  Services  (describe  services;  contract  costs)*3 
Base  Support  Costs*3 

•  Personnel  processing  and  administration 

•  Housing 

•  Food  services 

•  Fire/police  protection 

•  Medical  services 

•  Legal  services 

Personnel  Processing/Administration;  cost  of*3 
Overhead  Costs*3 

•  Command 

•  Administration 
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SUPPORTING  INFORMATION  FOR 
IMPLEMENTATION  PLAN  AND  ECONOMIC  ANALYSIS 


CBE  Program  of  Instruction 
Conversion  Hours  by  Course 


Course  Implementation  Sequence  by  Fiscal  Year 


Student  Flow  Scenarios  by  Course 


Annual  Cl  Student  Personnel  Costs  by  Course 
for  High/Low  Student  Flow  Scenarios 
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Table  C-9 

CbK  Section  Costs,  FYs  1979-81 


Cost  Element 

FY79 

FY80 

FY31a 

Military  Personnel  Services 

$158, 509 

$158,509 

$158,509 

Civilian  Personnel  Services 

82,285 

82,285 

82,285 

Supplies 

4,900 

6,700 

29,700 

Purchased  Services^ 

9 , 000 

13,300 

72,000 

■  Total 

$254,694 

$260,794 

$342,494 

Estimated  FY81  costs  of  supplies 
projections  developed  by  the  CBF. 

and  purchased  services 
Section. 

are  based 

on  budget 

Purchased  services  include  computer-leased  maintenance,  telephones,  etc. 


Table  C-10 


CBE  Course  Development  Material  Costs  by  Course, 


FYs  1979-84* 


Course 

Man-hour 

Conversion 

Factor 

FY79 

FY80 

FY81 

FYS  2 

FY83 

Total 

CCMC 

Low 

$374 

SI, 682 

Moderate 

533 

2,400 

High 

800 

3,597 

FDLC 

Low 

353 

$  88 

Mode  rate 

353 

127 

High 

762 

191 

BEC 

Low 

655 

1,637 

Moderate 

946 

2,365 

High 

1,419 

3,552 

FROC 

Low 

433 

$1,298 

Moderate 

618 

1,855 

High 

927 

2,780 

RFC 

Low 

170 

679 

$  566 

Moderate 

245 

980 

816 

High 

425 

1,700 

1,416 

CRRC 

Low 

713 

2,140 

$1,605 

Moderate 

1,027 

3,082 

2,312 

High 

1,541 

4,624 

3,468 

Total 

Low 

$374 

$2,690 

$2,328 

$2,690 

$2,706 

$1,605 

Moderate 

533 

3,854 

3,355 

3,682 

3,898 

2,312 

High 

800 

5,778 

5,088 

6,021 

6,040 

3,468 

limited 

data  provide 

3n  inexact 

basis 

for  estimating  material  cost 

;  that 

cost  estimates  may  not  be  reliable  enough  for  budgeting  purposes.  Based  on 
the  cumulative  data  for  FY76/77-78,  the  ratio  of  material  costs  to  military 
man-years  for  similar  courses  being  developed/revised  by  Navy  instructional 
program  development  centers  a  cost  factor  of  $120  per  man-hour  is  used. 
Source  data  used  to  develop  this  cost  factor  were  extracted  from  N423:WEN: 
vm  CNET  Itr  of  25  Jan  79  "Instructional  Program  Development  (IPD)  Cost 
Report  for  December  1978." 
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